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FOREWORD 



This report presents Phase I of a three-phase project called "Design of 
Training Systems/ undertaken in consonance with the requirements of 
Advanced Development Objective 43-03X, "Education and Training." One of 
the major objectives of the project is to develop tools for the effective 
management of training organizations. The tools will include mathematical 
models which will be the basis for computer simulation of significant 
portions/subsystems of the training system. This phase is a required pre- 
lude to Phase II which involves the development of mathematical models and 
their validation. Phase III covers the verification of the models which 
have been developed. 

Sincere thanks is expressed for the close cooperation of all elements 
of the Naval Education and Training System both within and outside the 
Naval Education and Training Command. This analysis would not have 
been possible without such assistance. 

Principal investigators for this study were Dr. A. Elkin and Mr. L. Duffy 
who performed the basic analysis and developed the flows and supporting 
discussion of Section VI. The parts of the report dealing with evalua- 
tion of candidate models were developed by Mr. H. Bellamy* Mr. R. Yanko, 
and Mr. S, Stasak (part-time). The strategic assumptions contained in 
the Appendix (Volume 2) were developed by Mr. R. Hallman supported by the 
IBM Advisory Group. * 

The Training Analysis and Evaluation Group project team complemented the 
contracted effort by establishing organizational interfaces and by pro- 
viding guidance. 
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I. INTRODUCTION 

A. STATEMENT OF THE PROBLEM 

The Naval Education and Training System (NETS) is currently subject 
to extensive changes both internal and external. External changes 
include a switch from expansion to contraction of the Navy as a 
whole, the switch to zero drafti and prospects of decreasing military 
spending, Internal changes include the consolidation of most educa- 
tion and training activities under a single command and the applica- 
tion of new techniques and approaches to training. 

These and other changes permit, and at the same time require, innova- 
tion and flexibility on the part of managers and planners within the 
NETS. However, successful innovation requires some means for assess- 
ing the probable consequences of change prior to implementation. 
Such assessment can be the judgment of a manager or planner based on 
accurate data concerning the system as it exists, formal extrapola- 
tions employing statistical or deterministic techniques or both. 

The problem addressed by the Design of Training Systems (DOTS) Project 
is to employ the techniques of system analysis, educational technology, 
behavioral science, and operations research to provide a set of tools 
for gathering data on the performance of the system as is and for pro- 
jecting the consequences of changes to the system. 

B. SIGNIFICANT EARLY DOCUMENTS 

1- Request for Proposals 

In October 1972, a Request For Proposals (RFP) was issued by the 
Naval Training Equipment Center (NAVTRAEQUIPCEN) , Orlando, Florida. 
This RFP was directed specifically at the problem outlined in 
Paragraph IA and requested bids for a three phase effort culminating 
in the development of a^set of mathematical models to be used by 
the manager/planner in the NETS, 

The suggested lengths and purposes of the three phases were as 
follows: 

a. Phase I: 10 months, develop a descriptive functional model 
of the NETS with Naval Education and Training Command (NETC), 
formerly the Naval Training Command, as its focal point. 
Determine the assumptions, approaches and methodologies to be 
tested in subsequent phases using predictive techniques. 
Establish a technological data base for mathematical model 
design. 
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•• b. Phase II: 15 months, select the sub-set of mathematical 

models to be developed, provide a data base for model testing, 
project hardware/software and personnel requirements for 
operational system implementation, and develop a model 
verification plan for Phase III. 

c. Phase III: 12 months, verify the models developed during 
Phase II using real -world data and operational situations. 

The RFP did not contain government estimates of required man-hours 
or specific educational qualifications for personnel nor were 
responders limited to the phased approach outlined above. 

2. Management Support Plan 

One month after the contract award date (1 February 1973) the 
contractor submitted a detailed Management Support Plan (MSP) for 
Phase I of the study. This plan contains detailed descriptions of 
the study team organization, Contractor/Navy interface, tasks to 
be completed during Phase I, site visit schedules, and study mile- 
stone charts. 

Figure 1-1 shows the DOTS project structure. The Contractor Study 
Team (Figure 1-2) receives direction and assistance from the 
Project Office Team (Figure 1-3). 

The IBM Advisory Group provides internal review and project guidance 
as well as access to other useful expertise within the company. 
The Advisory Group has participated actively in defining assumptions 
as to future trends for the NETS. 

The Navy Working Group provides points of contact for the contractor 
team to draw upon, guides study development by contributing ideas 
and recommendations, and from time to time, contributes on 
specific assignments. 

The Navy Advisory Committee provides points of contact outside of 
the Naval Education and Training Command and provides policy guid- 
ance on the direction taken by the study. 

On 14 and 15 March 1973, the study team, the project office and 
the working group held a joint meeting to review and discuss the 
MSP. During this meeting some changes to the MSP, noteably in 
the travel agenda, were suggested. These changes were incorporated 
into the MSP. 
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By including any mutually agreed upon changes to study 
methodology or emphasis in the MSP as updates, the MSP becomes 
a working document reflecting the current direction of the 
study. The MSP is updated on a monthly basis. 

The MSP was updated to include the Phase II support plan on 
1 August 1973. An overview of Phases I, II and III as they 
are currently being carried out 1s contained in Section IV of 
this report. 
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BACKGROUND OF THE PROJECT 

A. ADVANCED DEVELOPMENT OBJECTIVE (ADO) 43-03X 

ADO 43-03X was originally promulgated by CNO in 1965, The over- 
all purpose of the original ADO was to study the use of improved 
training techniques and training evaluation methods. However, 
recognition was given at that time to the need for such planning 
aids as requirements prediction tools and educational technology 
impact assessment. 

As currently constituted, the subtasks under Technical Develop- 
ment Plan 43-03X are more heavily oriented toward learning systems 
and their control. Subtask POIA, the Design of Training Systems 
(DOTS), stresses a step-wise progression from systems analysis to 
the development of computer based models of selected subelements 
within the NETS. 

B. THE DOTS PROJECT 

The principal Development Agent for the ADO Program is the Navy 
Personnel Research and Development Center. The Training Analysis 
and Evaluation Group (TAEG) , NAVTRAEQUIPCEN, is assigned responsi- 
bility for the project. 

TAEG provides the multidisciplinary talent required to oversee a 
project which covers such a diversity of activity in terms of 
educational technology, operations research and systems analysis, 
and psychology. 



II-l 



TA£G REPORT NO. 12-1 



III. WORKING CONCEPTS 

A. EDUCATION/TRAINING AS A SYSTEM 

The scope of any educational/training system wnether military, 
city/state, or private industry will be determined by the structure 
and needs of the organization served by the system. A privately 
endowed university is governed by a different set of conditions 
than 1s a maintenance training system for an appliance manufacturer. 
Each serves its particular sector in a different way and operates 
in a totally different time context. Yet there is commonalty in 
that both systems are reactive and the success of each will be 
determined* in the long run, by their ability to react appropriately 
to changing requirements. Both systems are governed by certain basic 
conditions: 

1. Who is to receive education/training? 

2. Where is the education/training to take place? 

3. What are the students to be taught? 

4. How are the students taught? 

5. What will the students do after completing training? 

6. How well will they do 1t? 

The degree of importance of each of these considerations differs be- 
tween the university and the maintenance course because of the pro- 
found difference in the nature and purpose of the two systems. In 
some cases the questions are almost trivial for one system but 
Important for the other. 

The Naval Education and Training System (NETS) must provide levels of 
education and training from recruits learning basic military courtesy 
to cadets engaged in an undergraduate college curriculum and senior 
officers addressing strategic geopolitical questions. While for most 
education or training systems some of the above named conditions may 
be trivial (e.g., where training is to take place may involve only 
room assignment), NETS operates in an environment where all six factors 
may be of prime importance depending ori the part of the system being 
investigated. 

1- Types of ind i viduals receiving education or training: from high 
school dropouts to PhD candidates. 

2. Location of training: at least 34 geographic locations, many of 
which have multiple facilities. This does not count Naval Re- 
Serve or NROTC training. 
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3. Su bject matter : reflects the same variety and scope as the 
students Input to the system. 

4. Instructional methodology: ell types ranging from conventional 
classroom to self-paced individualized including simulators and 
operational hardware. 

5. & 6. Training Application and Evaluation : both reflect the 

d1 vers fty and size of the system served by the NETS. 

In order to study so large and diffuse a system efficiently, 1t was 
necessary to establish certain definitional boundaries and relation- 
ships early in the study effort. 

1. Functional vs. Organizational Approach 

The first question to be answered was the extent to which a 
functional system description could be achieved without reference 
to the organization. A functional approach to describing the 
system was stressed in the original proposal and, 1n view of the 
rapid organizational changes taking place within NETS, appeared 
to be most promising. However, functions are performed within 
organizations and the personnel performing those functions tend 
to describe their Input and output activities in terms of other 
organizations, Furthermore, command level organizations are most 
often clustered at a particular location so that the Investigation 
of a single function (which crosses organization lines) requires 
visits to multiple locations. Therefore, regardless of the final 
form of the system description the initial effort was oriented 
toward learning the organization and determining those functions 
which took place within each organizational element. 

Considering the broad range of students served and subject matter 
covered it was unlikely that the NETS would prove to be a mono- 
lithic, unified system. Initial investigation showed that the 
NETS is, rather, a set of relatively independent training sub- 
systems defined by training objectives. While the great majority 
of these subsystems come under the charter of the Chief of Naval 
Education and Training (CNET) some significant training activities 
take place in other organizations. 

2. Bounds/Interfaces of the System 

Initially, the functions included within the NETS were assumed to 
be similar to those shown in Figure III-l. Organizationally, 
tnose functions included within the dashed line are part of the 
CNET organization. Those functions wholely or partly outside the 
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dashed line belong to other commands or bureaus. In Figure 

wherever either personnel or information cross the 
dashed line an interface between the education and training 
system and some other system exists. There are some training 
activities which do not take place under control of CNET. 
These are enumerated in Paragraph III B. 

Figure 1 1 1-2 shows the top level command structure under the 
Chief of Naval Operations (CNO) and also Indicates those 
functions which are part of the overall NETS but which are 
carried out by different organizations. With the exception of 
BUMEO the interfaces between NETC and the other organizations 
correspond to the interfaces shown 1n Figure III-1. 

NETC interfaces upward to CNO and laterally to the other commands 
and bureaus under CNO. The interfaces shown in Figure II 1-2 are 
as follows: 

a. CNO/NETC. It is through this interface that operational re- 
quirements are translated Into training requirements. The 
Director of Naval Education and Training (DNET) acting 1n a 
staff capacity to CNO provides planning and resource require- 
ments for subsurface, surface and air. CNO also controls the 
enlisted rating structure and hence the levels and types of - 
training services required. 

b. Naval Materiel Coirmand/NETC provides projected requirements 
for training to support new weapons systems or major procure- 
ments of new operational equipment. The interface exists 
primarily at the Training Plans Conference but some course 
curricula inputs are received from Ships Engineering Command. 

c. BUPERS/NETC. Under direction from CNO, BUPERS provides quota 
control services for recruit training, basic school training 
("A" schools) and advanced training ("B" and "C" schools). 
BUPERS also controls the number and, to the extent possible, 
the type of enlisted personnel input to the training system. 

d. Fleet Commands/NETC. The Pacific and Atlantic Fleets interface 
with NETC via the Type Commands and Commanders for Training - 
Pacific and Atlantic. This is a complex interface and is 
shown in detail in Figure III-3. In some cases schools them- 
selves are controlled by CNET. 

In order to describe the NETS functionally, it was necessary for 
the study team to visit each of the organizations enumerated above. 
The functional system description required that those areas outside 
the domain of CNET be analyzed at least to the extent necessary to 
understand their impact on the training system itself. 
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ROLE OF NAVAL EDUCATION AND TRAINING COMMAND 

The Chief of Naval Education and Training (CNET) heads all naval 
education and training programs except medical and some Fleet 
training. Included within this charter are: NROTC, the Naval 
Academy* the Naval War College, the Naval Post-Graduate School, 
and the Navy Campus for Achievement. The CNET charter Includes 
the full range of training activities previously described in 
Paragraph III A. 

However, as an organizational entity NETC contains representative 
samples of virtually all common types of training and education 
within the Navy so NETC became the hub of the study effort. 

Given the resources available for this study it was necessary to 
select some parts of this diverse system for more Intense investiga- 
tion. It was possible to exclude certain areas from the outset of 
the study. Professional medical training is sufficiently special- 
ized to warrant its own organization, therefore, 1t was eliminated 
as an area of concentration. Most officer training, with the exceo- 
tlon of air officers, is too scattered geographically to permit 
formal description readily. 

On the other hand, recruit and advanced enlisted training account for 
1n excess of fifty percent of dollar expenditures for training with 
flight training accounting for an additional twenty-five percent. 
This reason alone would have been sufficient to warrant concentration 
1n these areas but the number of personnel Involved and the concentra- 
tion of a number of training functions in a relatively small number 
of organizations also supported this selection. 

Figure II I -4 shows the primary functional commands under CNET. Of 
the five functional commands shown, COMTRALANT and COMTRAPAC can be 
considered functionally as the same type of organization (although 
differences do exist) and CHNAVTRASUPP does not deal directly with 
the conduct of training. Consequently, the primary thrust of the 
study could be directed at essentially three organizations: CNATRA, 
CNTECHTRA and COMTRALANT/PAC. The selected organizations are heavily 
training rather than education oriented and are more concerned (except 
for CNATRA) with enlisted, rather than officer personnel. However, 
given the cost factors plus the requirement outlined 1n Paragraph 
III-A-2 to study and describe training functions outside NETC it is 
unlikely that the scope of the study has been unnecessarily limited. 
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I V , OVERV IEW OF THE PHASES 

A. STRATEGIC OBJECTIVE 

In defining the objectives of Phase I, II and I I I > the strategic 
concerns and assumptions of the Naval Training and Education System 
were considered. A significant, strategic thrust 1s directed towards 
providing NETS management with an expanded decision making capa- 
bility for rapid and effective response to factors which Influence 
the conduct of training. By application of advanced management systems 
techniques to the NETS program, significant increases 1n both 
efficiency and effectiveness are anticipated. 

DOTS is a seed project with the composite intant of Phases I, II and 
III being to provide the definitions, descriptions, validations and 
recommendations leading to an Integrated NETS management system. 

B. PHASE I OVERVIEW 
1. Objectives 

Phase I provides the critical data base for Phase:; II and III, as 
well as any future activities pertinent to development of the NETS 
management system. . The key objectives of Phase I are to: 

a. Develop a comprehensive description of the current NETS 
covering Its functions, lines of interaction between functions 
and the management and administrative procedures/processes 
controlling the system. 

b. Develop a set of strategic assumptions describing the NETS 
environment of the 1980' s. These assumptions are to provide 
hypotheses regarding significant social, military, educational, 
technological, etc., areas having substantial Impact on NETS. 

c. Based on the results of objectives a. and b., develop recom- 
mendations leading to, and a description of, an Idealized 
future NETS in terms of the projected needs of the 1980' s. 

d. Develop a 11st of potential or existing computer based models 
that will enhance the NETS management decision making process. 
The "candidate" list will be supported by a data base including 
simulation techniques, model structuring techniques, and a 
survey of existing data bases and systems within NETS. 

e. Develop a Management Support Plan for Phase II and III based 
directly on the Phase I findings. This plan will be designed 
to assist in effectively utilizing DOTS study resources and 
in meeting scheduled milestones. 
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2. General Comments 

Although the major concern of the DOTS project 1s for the NETS 
management system, Phase I 1s Intended to provide a data base and 
recommendations pertinent to all appropriate areas. The descrip- 
tion of the system and the 1980's strategic assumptions should be 
definitive to a level that permits use by NETS as a "working" 
document beyond simply giving direction to Phases II and III. In 
any event, the Phase I definition and recommendations are essential 
to Phase II selection of computer models intended to significantly 
Improve the NETS management decision making and planning process. 

A detailed description of the approach used In achieving the Phase 
I objective 1s Included 1n Section VI. 

C. PHASE II OVERVIEW 

1. Objectives 

The output of Phase I includes, in addition to a definitive NETS 
description and general recommendations, a 11st of potential or 
existing computer models considered pertinent to assisting 1n the 
achievement of the strategic objective described 1n IV-A. Based 
on an assessment process, described in Section VII, the models 
were prioritized from the standpoint of scope and Intensity of 
. impact on the NETS decision making and planning process. Rased 
on this output as well as all other parameters provided by Phase 
1, the objectives of Phase II are to: 

a. Complete the assessment and evaluation of the candidate model 
list to assure that the list contains the most effective and 
definitive models pertinent to the NETS decision making and 
planning requirements. 

b. Select a model or modeling subset from the candidate 11st. 
The model selected should be sufficiently representative to 
demonstrate the validity of the use of models on the training 
planning and management environment. 

c. Design and develop the selected model (s) through coding and 
initial debugging. 

d. Identify and, where necessary, develop the data required for 
'casting and execution of the developed model (s). 

e. Complete level 1 validation of the developed model(s). Level 
1 validation consists of reasonableness testing, consistency 
testing, sensitivity testing, etc. 
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f. Design the on-site validation experiments to be conducted 
In Phase III. 

g. Develop a set of recommendations directed towards the 
future implications of model application and systems 
analysis to NETS. These recommendations will be directed 
toward estimating requirements for producing a NETS inte- 
grated operational system 1n terns of hardware, software, 
personnel, subsystems, data bases, etc. 

2, General Comments 

Although Phase II is primarily concerned with the application of 
mathematical modeling to predictive planning, its recommendations 
are not restricted, and, if appropriate, may include other tools 
of management systems. It is intended that, after verification 
during Phase III, the model(s) developed and validated during 
Phase II will be implementable as an operational Instrument 
within NETS. 

PHASE III OVERVIEW 

1. Objectives 

The primary inputs to Phase III are the developed and validated 
model(s), and the design for final experimental model verifica- 
tion. The objectives of Phase III are to: 

a. Verity the Phase II model (s) based on tests using digital 
simulation to exercise the model with the variables defined 
in Phase II and using real-world data. 

b. Provide a viable, productive and implementable modeling tool 
permitting NETS to effect cost and/or quality improvements. 

: c. Validate the recommendations of Phase II through extrapolation 
of Phase III experimental conclusions. 

d. Define recommendations that will permit future inclusion by 
NETS of new parameters, evolving from future educational and 
management research efforts, into the Phase III model (s). 

e. Define procedures permitting additional testing of the Phase 
III model (s) by NETS. 

2. General Comments 

The recommendations of Phase I, II and III are closely interrelated 
although each has some recommendations not directly impacting the 
other two phases. As was previously stated, the net resultant 
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should provide NETS with the guidance required to precisely 
define the additional projects required to achieve Its goal 
of a fully Integrated management system. 
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V. INTRODUCTION TO PHASE I 

A. FUNCTIONAL FLOW OF CURRENT SYSTEM 

The functional flow of the current Naval Education and Training 
System (NETS) 1s presented through a series of flow diagrams graphi- 
cally portraying the various NETS functions and defining their inter- 
relationships. Descriptive text Is provided where flow annotation 
1s Insufficient for clear understanding, or where expansion 1s re- 
quired. 

Problem areas and technological gaps in the current system are de- 
tailed in specific sections. 

6. IDEALIZED APPROACH 

An idealization based on the environment of the 1980's, is defined 
primarily through text and to a limited degree through additional 
flows. Most recommendations evolving from Phase I concern the im- 
provement of existing functions as opposed to major reorganizations 
of functional interrelationships. Both the current and idealized 
approaches are contained 1n Section VI for each NETS function. 

Although defined 1n terms of the 1980's, many of the recommended improve- 
ments are equally applicable to the current environment. 

C. ASSUMPTIONS AND VARIABLES 

To define the idealized NETS system, the problems, gaps and current 
functional flows were assessed against a data base consisting of: 

1. Assumptions and variables -leaved from the Phase I trips, Inter- 
views and documentation reviews. 

2. A set of strategic assumptions pertinent to the environment of 
the next twenty-five years. These assumptions were developed 
based on a number of strategic documents and from contributions 
of the IBM Advisory group. 

The working assumptions evolved from this assessment are defined in 
Section VI. The strategic assumptions are presented in Appendix A. 

D. TECHNOLOGICAL DATA BASE 

The technological data base 1s essentially a listing of sources, models, 
simulation techniques, data bases, etc., evolving from the Phase I 
effort. This listing is contained in Appendix B. 
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E. CANDIDATE MODEL LIST 

The candidate 11st of potential computer models was developed from 
an assessment of the Phase I NETS functional definition, assumptions, 
gaps and recommendations. In addition, data was provided by Inter- 
views conducted by systems analysts 1n parallel with those being 
accomplished to provide a data base for the functional description. 

Following the 11st development a procedure was developed and applied 
to the prioritization of the candidate 11st. The prioritized 11st, 
the procedure leading to prioritization, and ntcdel descriptions and 
selection rationale are provided 1n Section VII. 
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vi. 0escr1ptivs functional. mod el of the naval educ ation and training 
System ' " 

A. INTRODUCTION 

The previous section described the key objectives for the Design 
of Training Systems (DOTS) Phase I effort. The objectives in- 
cluded the preparation of a descriptive functional model of naval 
education and training as it currently exists, the delineation of 
assumptions that would be relevant to naval education and training 
1n the 1980' s, and the definition of an improved system which would 
more appropriately meet the needs of naval education and training 
In that time frame. Within the context of these objectives, this 
section presents the major functional elements of the current 
system. Then, on the basis of observed gaps 1n the system and of 
certain assumptions about the future, recommendations for creating 
a more cost-effective system are set forth. 

B. OBJECTIVES 

The objectives for this part of the Phase I effort are outlined 
below. 

1. Preparation of a Descriptive Functional Model 

There are many different kinds of models which serve different 
purposes in analysis, management, and research. A model can 
range from a series of tasks and decisions Interconnected and 
described in some symbolic art-form to a complex system of 
equations interrelated and depicted in mathematical form. The 
latter model is more easily computerized for rapid solution of 
highly complex problems. The former model has general utility- 
as a management tool, for gaining an understanding of the 
functional elements and interrelationships of a complex system, 
and for assisting in making selective improvements to that 
system. This management application 1s the basis on which the 
descriptive functional model presented in this section was 
developed. 

2. Limiting the Scope of Investigation to a Manageable Size 

The Navy is continually training to maintain a state of military 
readiness. Consequently, virtually every aspect of naval 
operations contains an element of training. However, time 
constraints did not allow a complete coverage of every training 
situation. Therefore, a general "sampling" approach to the 
analysis was mad-2. The primary focus was the Naval Education 
and Training Command. Even though the major concentration of 
Information gathering was within this command, data were collected 
from a great many other sources wh.ch are considered integral to 
the functional design of naval education and training. Among 
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these other sources were activities within tne Fleets, Bureau 
of Naval Personnel, Naval Materiel Command, Navy Recruiting 
Command, Bureau of Medicine and Surgery, Office of the Chief of 
Naval Operations, and Office of Naval Research. These organiza- 
tions are important in this study to the extent that they per- 
form functions related to the training system. This will hecome 
apparent as the functional description is reviewed. 

3. Delineating Gaps and Problems in the Current System 

There were two major reasons for conducting extensive inquiry 
into current and potential problems within the Naval Education 
and Training System (NETS). One reason was to help define areas 
where the application of computerized models would be of sub- 
stantial benefit. The other was to identify areas where tech- 
nological, functional » or operational improvements could be made. 
As previously mentioned idealized functional descriptions have 
been formulated to assist in the solution of these observed gaps. 
Those alternatives are oriented toward the 1980 time frame. 

4. Hiohl ightinrj Areas/Approaches Which Could be Improved 

The identification of existing gaps in the Naval Education and 
Trainina System, coupled with certain assumptions about trends 
in areas/factors which can be expected to influence this system, 
was the basis for the conclusions and recommendations included 
in this section. Important to understanding the system is the 
fact that relatively few operations which might be prescribed 
for an ideal system are totally absent in the current system. 
For this reason, the original objective of developing two 
separate descriptive functional models (one of the current sys- 
tem and another of the idealized system) would have led to 
needless repetition in the functional flows. In reality, the 
major differences between the two systems will be in the scope 
of application and in the degree of effectiveness with which an 
existing unction is carried out. This factor elevates the 
importance of the individual functional discussions where poten- 
tial areas of improvement are highlighted. 

C. ANALYTICAL APPROACH 

The analytical approach taken to achieve the objectives outlined 
above is described a^ follows: 

1. Field Visits 

A series of visits to activities both within and outside the 
Naval Education and Training Command accounted for a substan- 
tial portion of the data collection effort. An initial visit 
to tne Chief of Naval Education and Training (CNET) headquarters 
in Pensacola, Florida, helped to establish a general frame of 
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reference for the analytical effort to follow. Subsequent 
visits were scheduled at each of the functional commands re- 
porting to CNET (CNTFCHTRA, CNATRA, COMTRAPAC, COMTRALANT s 
and CHNAVTRASUPP) , at over 30 activities within these functional 
commands , and at over 20 additional training related activities 
within the Navy, A complete list of those activities visited 1s 
contained in Appendix D. Repeat visits were scheduled at each 
of the functional commands to obtain additional data, The final 
visit was a return to Pensacola which culminated In' Interviews 
with Vice Admiral Malcolm Cagle (CNET), his Principal Civilian 
Advisor, Dr. William Maloy, and other key personnel. The 
data collection was structured in the following manner, 

a. Methodology 

Interview outlines were prepared to guide the questioning 
process. Generally a single analyst conducted the ques- 
tioning of an individual, however, there were a number of 
exceptions to this procedure. The interviewed Individual 
was given a brief review of the project, its purpose, and 
the manner in which the investigation was being conducted. 
Every effort was made to insure that the individual under- 
stood the functional, rather than organizational, focus of 
the project. The idea of functional was conveyed, primarily, 
by means of a conceptual functional model (Figure III-l). 
The result was a high percentage of rather frank expositions 
of the data required. While the original Interviewed indi- 
viduals were selected according to a review of missions and 
recommendations of the working group, many of the subsequent 
interviews resulted from a need to follow up on a particular 
point expressed in a previous interview. 

b. Kinds of Data Gathered 

The data gathering focused upon: 

1) Identifying the major functions of training, 

2) Determining the important characteristics of these 
functions such as: 

a) Major inputs 

b) Major outputs 

c) Significant process steps 

d) Schedules 

e) Interfaces and coordination needs 

f) Problem areas 
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3) Classifying the functions into major functional 
categories 

4) Identifying important interrelationships between 
functional and subfunctlonal elements 

5) Identifying problem areas which detract from effec- 
tive functional operation. 

The objective of the functionally directed method of 

inquiry was to derive a data base for preparation of the 

descripUve functional model of the current Naval Educa- 
tion an<3rTraining System. 

2. Review of Naval Instructions, Reoorts, and Miscellaneous 
Documentation ' 

A number of instructions related to the mission of naval train- 
inq was requested for review at the outset of Phase I. Un- 
fortunately, many were not available to prepare for the initial 
data gathering visits in the most effective manner. However, 
many of these documents were obtained from the locations visited. 
Several reports such as "The Naval Training Command, A Report 
by the Naval Training Command Board," and "The Naval Training 
Command Executive Staff Studies" were available early in the 
project to assist in formulating the data gathering and general 
project approach. 

Literature searches were made through major information retrieval 
systems both within and outside of IBM. A substantial nu <§6r of 
significant reports and other documents were obtained in 
manner. The various reports, instructions, and other documenta- 
tion provided very useful reference in the development of the 
descriptive functional model. They were used as a framework 
for guiding the organization of the data mass collected during 
the field visits. 

3. Functional Flows 

The primary means of depicting the Naval Education and Training 
System is through a series of functional flow diagrams. 

a. Organization of Flows 

The functional flows are grouped in terms of the major functions 
of the education and training system. The functions are 
generally the same as those originally conceptualized and 
shown in figure III-1, however, some additions, deletions, and 
name modifications have been made. These functions are: 
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o Develop Training Requirements 

o Coordinate and Control Planning 

o Analyze and Plan Training 

o Implement Training 

o Evaluate Performance (Feedback) 

o Manage Training Resources 

o Perform Training Research 

o Support Training 

Within any one functional categoryi e.g., Manage Training 
Resources, a systematic exposition of the functional 
elements (subfunctions) is achieved by exploding the 
functions to greater levels of detail. Each major 
functional category is organized 1n the following manner: 

1) Definition. This introduces the function, defining 
its scope and major characteristics. 

2) Description. The description consists primarily of 
a written explanation of the function, a functional 
flow showing the first level of breakdown within the 
function, and an outline of subfunctions, sub-sub- 
functions, etc., down to the most detailed functional 
unit (tasks) contained within the functional flows. 

3) Current system. This section contains all of the sub- 
flows for each major functional category within the 
current NETS,. Where appropriate, discussions are 
included at the subflow level. 

4) Summary of major problems/gaps. This section sum- 
marizes the problems and gaps identified in the current 
Naval Education and Training System as related to the 
specific functional category being described. Only the 
must important problems and gaps are identified at this 
level. Such problems/gaps emerged as a result of 
observing or encountering the situation at a number of 
different locations. Several of the same types of 
problems were specifically mentioned by virtually every 
location visited. Other kinds of problems were developed 
as an outcome of analyzing the reasons for commonly 
observed problems and difficulties. 
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5) Idealized system. This section provides recommendations 
for increasing the cost-effectiveness of the NETS. These 
'take the form of specific statements of needs/objectives, 
and where appropriate include suggestions on methods of 
achieving these goals. Recommendations are based on 
problem areas and functional elements that require improve- 
ment, as well as on assumptions about the 1980'$. 
Originally this was conceptualized as a separate model, 
however, the analysis showed that very few functional 
elements necessary for an effective future system were 
missing in the existing system. The major area for 
potential improvement was in the effectiveness with which 
a certain function was carried out or in the degree to 
which it was applied in different organizational contexts. 
For this reason, this section will be primarily analytical. 
Separate flows are produced only where a new functional 
element must be included or where a different functional 
arrangement might be more effective* 

Characteristics of Levels 

Eacti succeeding subdivision of a function provides greater 
detail on the composition of the function. It also 
identifies functional interrelationships which are only 
generally apparent at the higher levels of aggregation. 
There was no intent to detail any function down to the 
procedural (how to do) level. The functional breakdowns 
are intended to describe what takes place, how activities 
relate to each other, and the interfaces extending beyond 
the NETS. In some cases, an additional more detailed 
level of breakdown would result in a procedural description ; 
in others, three or four more subdivisions might be necessary 
before the procedural level is reached. 

Annotation Methodology 

Each major functional category is designated by a number; 
e.g., 1.0 for Develop Training Requirements, <T.G for Implement 
Training, etc. Functional elements at the first level of sub- 
division would be designated 1.1, 1.2, 4.1 > etc. For each 
further subdivision of a functional element, an additional 
number is added; e.g., 4.1.1, 4.1.2, 4.1.2.1, 4.1.2.1.1, etc, 

Functional activities may be accomplished either manually 
(Manual Activity), automatically (Computer Activi ty) , or in 
some combination (Manual/Computer Activity). Differently 
formatted symbol oqy is used to define these various activities. 

Knowledge of the decision elements within a system is important 
tc understanding the system. Therefore, separate symbol ogy is 
used to show manual as well as computer assisted decisions. 
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For a function to operate effectively, a number of other 
functions may have had to be performed in advance. Logic 
symbol ogy i therefore, is introduced to clarify functional 
interrelationships that depend upon certain combinations of 
activities/events. 

Alternate or new functions proposed for the idealized version 
of the NETS have been uniquely identified, and the modified 
flow diagrams have been interspersed with those of the existing 
system's description. A function which is believed not to 
exist in the current system has an M N" (for New) preceding 
its number, and an "A" following it. If the function, already 
exists in the current system> but a modification is recom- 
mended in the idealized system, the current number will be 
appended with an "A", 

A detailed key to the symbology used is shown in Figures VI 
0.0 and 0.1. Note that minimal symbols are used. The aim was 
to avoid flow diagrams which require a programmer background. 

d. Elaborative Notes 

The functional flows in themselves, provide a fairly compre- 
hensive picture of the NETS. However, since further clarifica- 
tion was considered necessary in certain instances, notes have 
been integrated into the flows as required. 

D. GENERALIZED FUNCTIONAL DESCRIPTION AND ANALYSIS OF NAVAL EDUCATION 
AND TRAINING ■ 

1. Overview of the Current System 

In most respects the functional elements of naval education and 
training resemble those to be found in any education and training 
system. A technical subsystem carries requirements from the 
development stage to the implementation stage and through to the 
evaluation stage. A resource subsystem tracks the operations to 
fulfill these requirements through each stage. These two basic 
subsystems operate together for the purpose of regulating the 
flow of students through the education and training system. Thus, 
the similarities with other education and training systems are 
quite apparent, The purpose of this section is to provide a more 
specific description of that system which operates to control the 
flow of students within the Navy. 

A primary consideration in describing naval education and training 
is its relationship with the overall "Navy System. n In effect it 
is a system within a system and this fact accounts for many of the 
characteristics which make education and training within the Navy 
unique, at least when contrasted with the academic and industrial 



ERLC 



VI -7 



TAK REPORT NO. 12-1 



1 



h AC J 
O UJ CO 

* z h 

H V> D 
UJ O sA 6. 

wroz 
< & 

I 4 



■ 



* < & # 

>• X 

O * v*> h O * 
2 *- 

4 mJ \A 

V) < 
* ft 

X T. W 

• H < Ci ♦ 

r 



* 

« o 

# J- UJ H 

X V -4 

3 J t' 

u. <* u 

i 5 n 
o m z 

o x 

_ uj x h uj 
o # rr ui 

» < O) c 

u * au O — 



o • 



9 

ERIC 



«% « — — — — * 



H X O 
» H .J 

o u. 
h x 

5KHJ 
UJ * 
2 > M 
vi H H 

X > UJ 
1 ^Hm3 
I M *~ 5 

h H < \A i 

I 

— • 



I X o 

i:: 



• « 

'A' 



• — — ♦ 



+ 

UJ 

H 
o 



UJ 



♦ 
♦ 

UJ ♦ 

H ♦ 
o ♦ 
z ♦ 

UJ ♦ 

w + 
v> ♦ 
— ♦ 
♦ 

• ♦ 

♦ ♦ 
o # 



♦ * 

x : 

v> h ♦ 
o v ♦ 

♦ 

H O Q r- 4 
wiuZZ ♦ 

> X •* uj * 

C ♦ 

hM> & f 
O J J3 ♦ 

< o. ~* **• ♦ 
x < a ♦ 

o D uj + 

XO<0l ♦ 
t- < X o ♦ 

♦ 

f— 

<. ♦ 
a ♦ 

i- > < i_ + 
00 O D ♦ 

>■ a ♦ 

H o u. uv •> 

MOJO ♦ 

> z u> ♦ 
~ V> X Z ♦ 

0 «j O ♦ 

* ft. U. ♦ 

gUJ + 
UJ o f 

1 u O a: ♦ 
t- < \A a ♦ 



• — ♦ 



fx I 

UJ < 





i 




o 




o 


1 




UJ 


I 


X 


1 


> 


X 


1 




1 






1 


UJ 




1- 


m >■ 


I 


UJ 


1 


u 


mJ ~J 


1 




i 














K < 


j 








o ^ 




• 


f 


*-* 


U 2T 






1 


X 


U < 


i 




1 


r- 


< X 


1 


• 

O 


1 

4 






t 

• 




♦ 




i: 


# 


• 

o 


1 
I 






1 
1 


X 


t 






J 




\ 






1 


s- 


1 


X 


zoo 




u 


1 


u 


M M 


j 
















X 


X o >~ 




u. 






-j 














t— 




>- 


w> U. X 




-o 




^« 


UJ < 










O tu 








> 


X o 


1 








> X 


i 


w 


1 


»- 


o — 


i 




» 


u 


aC O u» 


1 


a 




< 


6. f- W 


t 


UJ 












• 






* 



Uf 

UJ 00 

x o 

X O 3 
« a. o 



♦ ♦ 

^ o 

♦ -4 UJ ♦ 

Nl X 

♦ W» m O 4 

M H 

UJ 

*/» H «M 

♦ D U ♦ 
X fX UJ 

♦ x a ♦ 
o 

♦ o ♦ 



o 

X 



z or 
< o 

u 

-J UJ * 

-J v> 

I UJ • 

O 

UN U Z 

M < 

UJ o 
K O UJ 
O «J «Q 

z 



z uj a 

O Of UJ 
nOCO 

r- u. X 

U UJ 3 
Z 03 Z 

UL O UJ 
Ul O 
31 U Z 

UJ < UJ 
Z «J * 

a uj 

• . & 
*?« 

UJ UJ 

t-ZX 
O < i- 

z 



UJ UJ fH 

Z l 

£5° 

J M 

rX > 
X 
• UJ UJ 

U uj O 
O X « 
v k a 
o « 

X 3 Z 
^ U. m 



z 

v> Z •* 
UJ v) • 

Q * UJ 

m z 

H O UJ 
< 5 O 3 

<o u. z u 

O » V* UJ 

xw< x 



Z Of z 




O Qt UJ < 




m UJ CO 






4 \k 3 Ul 


2 S 




OHM 


►* < 


•-•Ok- 


jouiu 








UiOZO 


003 


O < UJ z 


*•* u, 


*-* -J ot 


UJ 


CL UJ 


W Of O 


1 U. a 




I Ui« 


z > 


t* < u < 


Of 1- 


1 <M 


UJ O <A 


UJ UJ • 


K UJ M 


X X o 








z 





' o 

»✓> UJ 

^ u 

vA < 

~J 

^ a 

UJ 

iA \A 
< M 

Of 

i uj 



Of 
UJ UJ 

• X X 

Ohh 
Z Of 

•< u. 

u X 
Zhl*l 

3 *« 

U. WO 

uJ Z 
OHO 



N^l OH 

UJ X O Z 
H X UJ Z O 
0< X~ SL 



O 

4 -J 

u, 

z 

O UJ 

38 

M UJ 
M CO 

UJ 3 



l tu 

UJ o 
X < 

4 «J 

Z UJ ft. 
X 

h < UJ 

ozo 

D X -J 
U. O *J 

I H H 

u 
~* x * 

u> 2 O 

H wJ 
O < CO 

z 



51 -J < 
O -J UJ 

H -J X 

u. u. x 

UJ l 
JZU1 

o a uj 
Of < o x 

UJ UJ O H 

a m 

a. u> u> o 
DO UJ 
UJ «J 
UJ Of Z H 
X Ui h m 
HZ H 

0C O 
X O X UJ 
mU<0 



! 



-J 

u. 



i 
I 

o 



UJ 

! 



Vl-8 



TAEG REPORT NO, 12-1 





r 


or 


l 




«r 


UJ 


» 






X 


l 






• 


i 


X 


1 


M U 


l 


M 




Ui • 


l 




% 






1 




iL of o 


1 


of 


X 


M O UJ 


1 


o 


o 


at 


l 


1 




Uj • at 


1 




fr- 


Ut3 3 


1 


fr- 


u 


* ■ o 


1 






-J u 








a. x a 


1 






ft 






UJ 


Ui M UJ 


• 1 




fr- 


w fr- > 




x 


< 




» 1 




U 


►- « X 


& ft 



I 

uj 
-J 
a. 

X UJM 

* v K 

X <I l ) 
UJ < 



X 

O 



I 

UJ 

a o 

< fr- 
K < O 
UJ W 4 



a 



V 



Oft-* — — i 



i 



i 



-J X 
a o 

X UJ ** 
« * fr- 

x < u 

UJ fr- * 





JJ a 














fr- x 














< i * 
















4 












~ i 

Q UJ 


1 

«« 












r >- 


• 










UJ 


•m X u. 












-J 


< < 


u. 








■ J 


a 


ot 


r> 








« 1 




UJ f of 






1 




• 1 


o 


* 00 t> 


fr- 




UJ 






o 


?I o 


-J 




_J 




x < 




•< a 


t) 




6. 




o ( 




x x o 


VI 




X 








o 


UJ 




< 


* 


t 


o 


lA ++ X 


* 


■ | 


X 




O 1 


o 


M fr- fr- 






UJ 


fr- 


* i 


-J 


r o o 




• j 








a 


»- < o 


< 


o ft 




"5T 


— • 



Of Of 

UJ o 
X 

t fr- » 

•f >- I 

0 Ui UL 
V • 
ti fr- 

;e « cc 
*< x a 

1 h 

i 

►» ^ w 
n uj t 
a. 

>- * r 

O n 
4 «f 



UJ r; 
of ad 
< a. 

• UJ 

U. ftf 

t a 

u, 

ut 

Q BO 

• Q 

UJ Ui 

t Of 

x n 
r» 

o UJ 
CO * 



• 

fr- * 
X < 
UJ • 



uj n 

X M 

O 
o «« 
fr- 

X 
o o 
x n 

■M fr- 

o u 
ui x 

UJ !3 

o u. 



fr* of 
X < a. 
HJI 

»« ►« «"» 

x r 

M X 

Jft W) «c 

Of 

0 < n 

V < 

Q X 

< UJ 

1 o ^ 

uj X 

3* l/> 

«< r> x 



i 
i 
i 



<sj i 



i 

UJ 

wJ 

* I X 

O I * 

I X 

U. I UJ 
UJ I 

e* ft — 



• — ♦ 





i 
f 














1 
1 














1 
1 
» 














1 




1 1 






t 




J 




UJ 1 






a. 




} 
1 


1 


• 1 




1 


• 




1 


UJ 
•J 


5? 1 




UJ 


X 






a. 


a ( 




& 


a 




J 


X 


UJ w, | 




X 


Ui 




? 


< 


V } 


*■> 


* 




• 






« U 1 


1 


H 


^ o 




1 


UJ 


>- 4 1 




UJ 


I- < 


• 


f 






» 







or 
C5 



tA UJ 
UJ <0 













• 


X 




^ »- 






Ui 


X — 


UJ 




o t/» 


UJ 


X 


X 




X 




1-4 3 




o 


fr- 


• 


• fr- 


• 


O X 




c> 




>• M 


et 


• 


X 




o 


it. 


fr- ■ 


a ui 


• 


\A 


or 




o 


M O 


• 


• 




o 






> w 


fr* 


• 




u> 


UJ 




K4 


fr* < 




a a. 




> 


X 


UJ 


r> x 




X X 


L~ 




UJ 


u o 


ev fr- 




< •< 


X 




X 


* X 


►- 




• 


UJ 


ii 


X 


Ui 






X 


1^ 


UJ 


to 


>- D 


O Ui 




UJ *- 


Ui 


u. 


M 


x a 


fr- 




X o 






^i 


UJ Ui 


X < 




»- ft) 




Ui 


0. 


x w> 


* o 



I 

o 

X SA I 

i < < 

u» o X 

• o o 

X « 
X * w 

O fr- L> 

O aC < 

• A Ui 

Ui fr. u. 
8 fr- U* O 

a < 

XX fr- 
O ^ 
fr- ^ 3 
J- 3 O 
OAUU 

< — o * 



o — 

M« UJ M 

> X 

ui 

- «- > i- 

«rf a. -j x 
- « x < • 

• • O 3ui I 

o ♦ o X ui • 

ft TU4^« 

u. ft fr- « X - • 

UJ • • 



• I 

o * fr- 

X \ t> 

\ a. x 

fr- I X X O Q 

UI I UMMtiJ 

"* M fr- *J 

X X o > 



u, ^ 

fr- V- U_ 

n fr- u,' 4 

a. Hfioj 

x i > x o I 

~» ) N* > fr- T I 

w I fr. O Mi 

I OtfOUJ I 

u i * a. fr- « i 

u* » i 

of * — — — — « 



ft 




1 


t 
f 


fr- j i i 


uj a Cs 


1 


1 Q X O 4 


x ui ui x x 


1 


uj X < -J 1 


< X X * +* 


1 


O U. 1 


X < fr- of X 


1 


X fr- O I 


X U» Ui fr- 




- 3 fr- X 


fr- X O X M 




a. uj 


!> X O < fr- 31 




X X Q fr- 


x a of a. of 




< M UJ l/) 






O V 


u. n» ouluji 




(Ah f krt 


o x x o n 




M X ID 


l ar w w w m j 




UJ ft* Of | 


3 W O A > U. 


i 


sA O ui uj 1 




i 




M U -J O SA 




X uj ui fr- | 


UJ UJ UJ JH)M 




fr- a oe o i 


fr- x v» of r» x 




* 


O fr« •* a. 31 lA fr- 


♦ 


ft 


X 



VI -9 



TAEG REPORT NO. 12-1 



systems. It 1s the nature of the overall Navy System (which takes 
its character from higher level systems such as military, defense, 
government, and political), that tends to shape the character of 
the NETS. 

The personnel, materiel, recruiting, operational (Fleet) and 
resource areas are primary interfaces with the NETS. The descrip- 
tion of the current system, the identification of some of the 
major problems/gaps, the assumptions made relevant to the 1980 
time frame, and the recommendations which form the basis for 
system idealization must consider the contextual relationship 
of the NETS to the overall Navy system. 

The personnel system drives training in two major ways: (1) 
it promotes the need for occupationally oriented training found 
in the "A" schools, and (2) it influences the availability and 
quality of training-staff personnel. 

The materiel system drives training by the way that it procures 
new weapons systems and by the way that it handles subsequent 
modifications and retrofits. The procedural and operational 
relationship of the materiel system to contractors has a heavy 
influence on training. 

Recruiting determines both the profile of new student inputs to 
recruit training and to some extent influences the need to provide 
additional occupationally oriented schools for reenlistees ("B" 
schools) . 

The operational system is a continual driving force in terms of 
recurring training needs (numbers of students) and in terms of 
tactics/doctrine/operational changes as they might modify course 
content. The impact of resources and the resource management 
system are similarly pervasive forces within the NETS. One of 
the basic requirements for coordination and control (management) 
stems from the fact that the available resources are almost always 
less than those desired. 

Another factor to consider is the very mass of the Navy System. 
Many of the problems observed are a direct result of the overall 
system's inertia. This is especially true in terms of obtaining 
personnel to handle training requirements which have not been 
well defined in the planning/programming cycle. Many of these 
types of requirements are for the purpose of improving cost- 
effectiveness, therefore, lack of flexibility results in excess 
resource usage. 

The specific functions and functional interrelationships observed 
in tne data collection and analysis phase of the study are shown 
in Figure VI 0.2 - Naval Education and Training System (Informa- 
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tion and Control) and Figure VI 0.3 - Naval Education and Training 
System (Personnel/Trainee Flow). Two additional major functional 
areas were identified during the study which were not a part of 
the original conceptual model (used as a framework for the data 
collection, Figure III-l). 

The function, Coordinate and Control Training, recognizes manage- 
ment as a separate and distinguishable area. A number of 
individuals perform this function as the major part of their job, 
others perform it in a much lesser capacity. Nevertheless, 
it is a necessary and primary function for achieving the 
desired level of operational effectiveness. 

Another function, Support Training, has been separately identified, 
again, because of its importance to the effective operation of the 
total NETS, 

In summary, the current NETS is composed of a primary technical 
subsystem which has significant interfaces with other major 
components of the entire Navy System. The main components of 
this subsystem are (1) the development of training requirements, 
(2) the analysis and planning of training, (3) the implementation 
of training, and (4) the evaluation of performance, The function 
of training support primarily augments the training implementa- 
tion function. Another major function within the technical sub- 
system is training research. This function is coupled with per- 
formance evaluation in terms of research results which improve 
evaluation methods, and is tied to the training support function 
in terms of research results which lead to improved training 
technology. The management of the system is described through 
the function of coordination and control which has its primary 
influence on the function of training analysis and planning. 
Finally, the training resource management subsystem is primarily 
concerned with defining the resources required to accomplish 
training plans so that they can be implemented effectively; and 
secondly, with controlling the expenditure of resources once they 
have been allocated. 

A number of specific problems are identified within the individual 
functional descriptions and analyses which follow. These can be 
summarized into four major areas: 

a. Interface Problems 

This class of problems relates to training difficulties which 
arise because adequate communication, coordination, liaison, 
or other interrelationships with areas both in ard outside of 
the NETS have not been well defined or implemented. 
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Examples of interface type problems are: 

1) The right people not attending a NTPC. 

2) The existence of different bases for training and job 
advancement. 

3) Ineffective trade-off analyses between equipment design, 
the personnel assigned to operate/maintain the equipment, 
and the training program. 

4) Inadequate definition of training research requirements 
to the research laboratories. 

5) The redesign of feedback methodology by multiple training 
activities. 

6) Inadequate definition of Shore Establishment Relocation 
(SER) impacts on remaining training activities. 

7) Course length and content being different at different 
training locations. 

Personnel Problems 

This class of problems relates to training difficulties 
stemming from inadequate selection, assignment, or development 
of staff or student personnel within the NETS. 

Examples of personnel type problems are: 

1) Assignment of some poorly qualified (promotional risks) 
personnel to management within the training system. 

2) Maintaining continuity of effort in key areas in the 
face of frequent personnel rotation - 

3) Orienting training to the specific billet (job) require- 
ments. 

4) Obtaining skilled manpower to accomplish important R&D 
studies related to improving training effectiveness. 

5) Obtaining an accurate match between the amount and type 
of training and the requirements of the job. 

Information Problems 

This class of problems is related to deficiencies 1n appro- 
priate data for making sound decisions with regard to training. 
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Examples of Information type problems are: 

1) Inaccurate and varied data from multiple sources. 

2) Difficulties in accurately projecting "C" school require- 
ments because of NEC data baso. 

3) Inconsistencies or total lack of time distribution data 
at the level of detail required for cost allocation. 

4) Inadequate definition of certain terms such as "average 
on board" or "school." 

5) Ineffective task analytic data available for definition 
of course content. 

6) Insufficient technical data base for decisions to reduce 
or eliminate courses. 

7) Inadequate data base on which to systematically develop 
an Integrated long-range training plan through all 
command echelons. 

Resource Problems 

This class of problems is associated with training difficultly 
which arise because resources are not available at the right 
time, 1n the right place, or in sufficient amounts. 

Examples of resource type problems are: 

1) Insufficient PCS funds to accomplish cost-effectiveness 
programs . 

2) Inadequate definition of implementation costs for pro- 
posed projects. 

3) Obtaining timely detailing of personnel to accomplish 
unplanned, cost-effective projects. 

4) Difficulties in assigning priorities to requirements 
needing resources. 

5) Obtaining adequate software support for training devices. 

6) Obtaining reasonable match between the technical and 
resource requirements of a urogram or project; e.g., 
Civilian Substitution Program (CIVSUB). 
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It must be emphasized that these are generalized system's 
problems which are not aimed at any one organization, but are 
present in the totality of the NETS. Any of the problem areas 
covered here and in the discussions for each function must be 
viewed in a total context by each organization to see to what 
extent the problem pertains. 

Strategic Working Assumptions Expected to Influence the 
Education and Training System of the 1980 1 s 

The recommendations contained in this report are pertinent to 
the current Naval Education and Training System and are also 
intended to address the needs of the 1980 's . This section 
contains a set of Strategic Working Assumptions (SWA) forming 
the predictive assumption base considered in developing the 
tactical assumptions and recommendations of the functions to 
follow. 

The SWA's represent a synthesis of a more complex set of 
Strategic Assumptions (SA). To reduce the complexity of 
Section VI and permit emphasis on the two key objectives of 
definition and recommendation, the strategic process definition, 
Strategic Assumption Charts, and SA Interaction Charts are 
contained in Appendix A. The Interaction Charts permit a direct 
correlation of the synthesized SWA's to the more detailed SA's. 
The SWA's represent the key futuristic environmental definitions 
impacting the NETS of the 1980's. The following points are 
essential to an understanding of their use in this report: 

a. The SWA's are based on SA's fixed at a point in time and 
should be considered transitory. No strategic process is 
valid that does not provide for frequent reassessment of 
its assumptions. 

b. The tactical assumptions and recommendations of the Section 
VI functions are intended to support the strategic direction 
implied by the SWA's but also consider other parameters. 
Therefore, there is no one-to-one correlation of SWA's to the 
tactical elements of the functions. 

c. The strategic process and SA's of Appendix A should be 
considered as the starting point for a NETS strategic 
process and not as a final definitive document. It does 
provide the rationale for the SWA's essential to this 
project. 

d. The SA, SWA and Tactical Assumption approach was incorporated 
to permit a logical flow from a complex set of futuristic 
interactive assumptions to the lower level of tactical 
recommendations. 
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The Strategic Assumptions are d1vie.'<id Into the significant 
categories of Demography, Philosophical, B1o-Medical, Social/ 
Cultural, Communications, Education, International, National, 
Domestic Institutions, Technological , Military Doctrine, and 
Naval Factors. Uly those SA's felt to have some significance 
to NETS were developed. Obviously, they could be expanded under 
the above categories to support functions other than education 
or training. 

The Strategic Working Assumptions synthesized from the SA's based 
on their pertinence to NETS represent an additional step 1n or- 
ganization evolving from the SA categories. The resultant pro- 
vided practical guidance to recommendation development. 

The major SWA divisions pertinent to NETS are: 

o Resource control 

o Resource distribution 

o Human resource 

o Organization 

o Process 

Refer to Appendix A, Section II D, Page 87, Strategic Assump- 
tion Structure, for the networks linking each Strategic Assump- 
tion to the key driving Strategic Assumptions. 

SWA statements and supporting comments are: 

a. Resource Control 

Statement 

A confluence of factors will result in a significant 
reduction in the resources available to accomplish the 
NETS training mission. Although a current concern, the 
degree and scope of resource restriction will become 
increasingly severe through the mid-1980 's and will result 
in major changes to the NETS resource control approach and 
structure. 

Comments 

Economic necessity will dictate a reduction in the total 
resources required to accomplish the training mission as 
well as a reduction in the unit cost of training for any 
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given task. All command levels of NETS will experience 
increasing levels of Inquiry and control from higher govern- 
ment sources. The following change statements, are pertinent 
to this reduced resource availability: 

1) There will be an Increased capability for cost effective- 
ness quantification, measurement and control. 

2) More systematic and comprehensive cost justification 
methods, techniques, programs, etc., will be developed 
and implemented. 

3) There will be an improved capability for marginal 
cost identification. 

4) Increased application of computer technology will 
result in a need for more uniform and consistent data 
bases . 

Resource Distribution 

Statement 

The projected combination of restricted resources, technolog- 
ical change, national doctrine, and military strategy will 
result in a major change in the traditional distribution of 
resources across the military services and, also, across the 
various commands and functions of any given service. The 
impact on NETS will be substantial since its mission 1s driven 
by most of the other functions within the Navy. 

Comments 

At the service level, increased mission sharing or consolida- 
tion will become desirable and essential to the maintenance 
of national doctrine and fulfillment of military doctrine. 
Technologically, the need and potential for the optimization 
of the various technical functions will mandate shifts 1n 
resource utilization that will influence NETS. Pertinent 
change statements are: 

1) Significantly increased interservice sharing of training 
resources will become a major factor. 

2) Increased uniformity between training locations within a 
service or between services will be achieved and controlled. 

3) Improvement in the planning process will result -in 
optimal distribution of training modes within the total 
training system. This will result in a trend to more 
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on-board training based on self-study modes. Self-study 
encompasses all forms from relatively unstructured train- 
ing sequences to Computer Assisted Instruction (CAI). 

4) There will be an improved capability to optimize resource 
distribution through appropriate trade-off's between 
personnel, equipment design > and training factors. 

Human Resource Management and Accounting 
Statement 

The average recruit entering the military service In the 
1980's will have been Influenced by the changes in national 
culture, attitudes and the total pre-service education process. 
The changes resulting from these influences combined with 
the changing requirements of the services will necessitate 
changes in the management of human resources, especially 1n 
those areas leading to a trained and motivated person capable 
of fulfilling his designated mission. 

Comments 

Since the assumption of decreased resources implies fewer 
people to accomplish a specific mission, it is imperative 
that the utilization of human resources becomes more efficient 
and that future changes 1n personnel profiles be considered 
1n designing the NETS of the 1980's. The steps leading to 
improved human resource management and associated accounting 
will support changes projected in the NETS. Pertinent change 
statements are: 

1) There will be a need for increased emphasis on attitude 
shaping. Also, behavior modification will be required 
in those key areas where a compromise between Navy needs 
and individual characteristics, goals, or motivations 
cannot be made. For example, a recruit conditioned by 
his or her pre-service experience in an Individualized 
training environment, permitting maximum freedom of sub- 
ject and objective selection, may have difficulty in 
accepting a set of optimized objectives that are mandated. 

2) The All Volunteer Force (AVF) has already resulted in re- 
cruit profile modifications. These changes will continue 
to be felt through the 1980's due to the continuing input 
of new recruits and the extended influence of the AVF 
recruit coming aboard today. 

3) The resultant of a need to maintain force strength through 
the AVF, to adjust to a changed recruit profile', to 
accomplish missions efficiently, and to capitalize on more 
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efficient training approaches will be a major revision, 
or revisions, to the Navy occupational structure prior 
to the mid-1980' s . A much closer match between career and 
billet needs (job performance requirements) will be 
achieved with career growth taking place within a well 
defined but flexibly structured framework. 

4) Computer technology will be applied to the Inventory 
maintenance of Individual capabilities and training, 
and to the subsequent use of these data in meeting mission, 
training, and career needs. 

The need for this capability will become more essential 
to NETS as the individualization of Instruction increases. 

d. Organization 

Statement 

Inter-service cooperation and consolidation of the training 
function, increased need for precise control of training 
resources, a need for human resource accounting, the thrust 
towards instructional modes requiring highly systematized 
and disciplined development, and a shift to more on-board 
training will result in a NETS exercising strong centralized 
control over all Navy education and training by the mid-1980' s. 
This statement is predicated on the attempted implementation 
of similar changes by other organizations lacking strong 
elements of central control. Increased control will be 
mandatory if the education/training mission is to be accom- 
plished as currently planned. 

Comments 

Although prior statements imply a significantly increased level 
of training system control, the development of the required 
organization to accomplish this change is so critical to the 
future achievement of objective? it 1s included as a separate 
statement. The thrust towards a higher degree of centraliza- 
tion and control of the total military training process will 
be given Impetus by the following change statements: 

1) There will be an organizational structure and process 
enabling precise fiscal measurement and control of the 
entire military training process by the mid 1980 's. 
It is probable that the present concept of separate 
ecucation/training functions for each service will re- 
main but this does not preclude the degree of reorganiza- 
tion required to achieve this postulate. 
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2) The capability of measuring actual job performance and 
of relating it to the training process will be developed 
by the mid 1980' s. This will assist in reducing the true 
cost of training and will make command accountability 
both possible and desirable. The complexity of the 
measurement process, the need for consistency, and the 
desirability of accountability will mandate an increased 
organizational control of all components of the Navy 
education/training process. 

3) Resource restrictions will mandate that increased cen- 
tralization of certain training functions be re- 
quired to achieve the economies of size. Some examples 
might be media development and production, editing, 
training evaluation, R&D, etc. 

4) Consolidation of training will be required if the current 
trend towards individualization of instruction leads to 
the use of computer teleprocessing nets to support CAI , 
CMI, CCI, or other forms of computer supported study. 
This 1n turn implies a need for organizational changes 

to achieve the justification and implementation of 
these nets. The same rationale applies to the use of 
teleprocessing nets for administrative control. 

Process 

Statement 

The present trends in increasing the educational efficiency 
and effectiveness of the NETS training process will continue 
and will be complemented by major improvements in the areas 
of job and task analysis. Improvements in these technologies 
will significantly increase the integration of the training 
process into the actual job stream especially in the main- 
tenance area and similar complex task environments. 

Comment s 

There will be continuing improvements in presently identified 
educational technologies. Application of thesu technologies 
will become increasingly desirable and practical as the cost 
of various delivery systems, simulators, and other training 
support devices declines and the systems themselves become 
more portable. Pertinent change statements are: 

1) By the mid-1980's, the majority of training will be de- 
livered at job locations using some form of individualized 
instruction and either a self-contained delivery system or 
one driven by teleprocessing rets. 
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2) There will be an increased emphasis on student motiva- 
tion in the self-study environment, and by the mid-1980's, 
gaps in this area will have been more clearly identified 
and techniques developed and implemented to eliminate 
them. In short, educational technology will be more 
balanced in the key areas of design, development, delivery, 
and human factors. 

3) Certain skills will be taught on-site as needed. The pre- 
viously mentioned improvements in educational technology 
will make this possible. These changes will complement the 
development of training/performance aids and the combinatory 
direction will significantly reduce traditional training 
requirements. 

3. Overview of the Idealized System 

The current NETS is performing every function shown 1n the 
overall descriptive functional flows (Figure VI 0.2). Conse- 
quently, no major functions need be added to idealize the system. 
Potential areas of improvement emerge in manner and detail. Most 
of what should be done is being done, although the way in which 
some operations are being performed may be made much more effective. 

Idealized approaches for each function are covered in the dis- 
cussion for that function. Since sequences and interrelationships 
are not involved as a rule, idealizations are presented as recom- 
mended goals/objectives and alternative approaches. Several specific 
research problems are called out also on the principle that, where 
necessary, idealization requires answers to fundamental questions. 

Specific concepts for idealization are covered within the dis- 
cussion of each function. This overview will touch upon those 
which are pervasive throughout the NETS and which are considered 
to be of the greatest significance. 

a. There is a need to increase the professional approach necessary 
to conduct a viable education/training system. There are some 
notable exceptions, but for the most part, a training assign- 
ment is regarded as an added duty outside the mainstream of an 
individual's career. Very often the performance of such 
duties reveals the secondary nature of the interest and capa- 
bility. A training career field and professional preparation 
is needed. The ideal goal is to develop training assignments 
into career stepping stones. 

b. There is need to increase the extent of long range planning 
in order to widen the planning horizon as well as to increase 
the sensitivity of NETS elements to future needs. 
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c. A stronger continuity of effort must be developed. The length 
of tours and the dearth of long term personnel is a clear 
problem. Longer tours for key personnel, civilian and 
military, are needed. 

d. The overall inertia of the total Naval System imposes a severe 
constraint upon the NETS, Within the NETS major organizations, 
a method is required for rapid shifting of assets, either 
temporarily or permanently, to meet unprogrammed loads, If 
peeks and valleys of workload can be known in advance, assets 
for such situations may be preplanned. Such flexibility 
should allow the manning of schools to handle average loadings, 
rather than expected peak loads as is the current practice. 

e. The move toward individualized self-paced instruction 
should be accelerated and standardized. The Naval Education 
and Training Command is working under a policy to head in 
this direction. However, the rate of such change 1s highly 
variable within individual commands/activities. Definitive 
guidelines and milestones must be established, Overall, 
there might be agreement that not all training is amenable 
to individualization, However, the exceptions should be 
specified. Reasons for exceptions are: 

Cost-effectiveness 

Rapidly changing technology 

Need for supervised hands-on training. 

Exceptions should bt well documented, and research oriented 
toward developing techniques of overcoming such problems 
should be instituted. Individualized instruction is used 
very little in naval training outside of the CNET command. 
Since from the standpoint of the total Navy system all train- 
ing should be maximally cost effective, those organizations 
not aimed toward individual instruction should examine that 
approach. 

f. The extent of technical coordination between elements of the 
NETS must be increased, This coordination refers to pro- 
fessional/technical interchange at all levels and between all 
commands in and out of the Navy, or in the CNET Command. 
There is a rteed for ittea interchanges in the form of meetings, 
news bulletins, and informal discussions. 

g. The extent of student loss through attrition is unacceptably 
high and must be reduced. Research upon the basic reasons for 
such fallout is essential. The er.d objective of such research 
will be techniques of reducing attrition and for developing 
cost-effectiveness guides/curves which will show the point of 
diminishing returns. Tne latter will indicate when further 
effort to "aid" the problem student is no longer worth the 
cost. 
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The approach to trairiihg research requires integration. The 
current fragmentary approaches do not yield the best results 
for the expenditure of funds. Central control under the lead- 
ing element in naval training (CNET) is essential. This must 
include systematic definition of problems, assignment of 
priorities, and prior commitment to implement effective results. 
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PRIMARY FUNCTIONS OF THE NAVAL EDUCATION AND TRAINING SYSTEM 

This section contains a detailed description and analysis of each 
of the eight major functional categories found operative within 
the Naval Education and Training System, The general outline for 
this section was presented in the description of Functional Flows, 
Section C3. 

1. Function 1.0 Develop Training Requi remen ts 
a. Definition 



To an extent this function is outside of the over-all train- 
ing system, since only needs for training are involved. It 
is the responsibility of the training system to fill these 
needs. However, there is no clear-cut distinction for this 
important interface. If needs are not perfectly clear, the 
training that is developed may not meet the need. The need 
will still exist and the training cost and time will be 
wasted. Because such situations can occur, and indeed have 
occurred, this function is being handled as 1f it were an 
integral part of the training system. 

An essential aspect of developing training requirements is 
the determination of exactly what is needed. Such determina- 
tion is performed by a subnotion called "Evaluate the Re- 
quirement." These operation? require an interaction between 
the training community and the originators of the requirement. 
Such interactions tend to be very rare in the current situation. 

Once the need is carefully defined in specific terms, an 
analysis and plan for training is developed (Function 3.0, 
Analyze and Plan Training). 

The development of training requirements encompasses all situa- 
tions which directly or indirectly impose a need for some type 
of training. Such situations fall into two broad categories: 

1) Develop training requirements for new systems, modifica- 
tions, or retrofits. As changes are made in hardware, 
techniques of operation, technology or maintenance, etc., 
these would tend to necessitate that personnel be trained 
in accordance with such changes. The cycle for developing 
requirements for personnel and training is described in 
NAVSHIPS 0900-060-0380 Ship. Life Cycle Management Support 
Manual - Plannin g Guide For Training Support of sTiip 
'Introductions and M odificatio ns , 15 August 1972, Naval 
SFip Systems Command, Code 047. 
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The sequence and Interrelationship of activities is 
relatively well defined In this guide and in OPNAVINST 
1500.8G, which in part is incorporated in the guide. 

2) Develop requirements for on-going training. Such re- 
quirements are due to changes 1n such things as personnel 
requirements/characteristics/input rates, etc.; changes 
in operational demands; changes in policy; as well as any 
other type of change which might indicate a new or modified 
training need. 

Description 

The Develop Training Requirements Function is described 
through an outline which contains all of the functional 
elements presented in the functional flows, and in addition 
through an overall functional flow, Figure VI 1.0 - Develop 
Training Requirements. 



Page 



i Develop Training Requirements 27 

.1 Develop Requirements for New Systems, Modi flea - 



.0 
1 

tions, Retrofits 
1.2 Develop Requirements for On-Going Training .... 30 

1.2.1 Develop Requl^ments Due to Change in Policy . 32 

1.2.1.1 Receive New or Revised Policy 

1.2.1.2 Estimate the Impact Upon Training 

1.2.1.3 Promulgate Policy 

1.2.1.4 Develop Preliminary Concepts 

1.2.2 Develop Requirements Due to Change in Opera- 
tional Demand 34 

1.2.2.1 Receive Information Relating to Change 
in Operational Demand 

1.2.2.2 Estimate Type and Extent of Impact Upon 
Training 

1.2.3 Develon Requirements Due to Changes in 
Ratings/NEC 36 

1.2.3.1 Receive Information Relatinq to Change in 
Rating/NEC 

1.2.3.2 Develop Training Requirements for New 
Ratings/NEC 

1.2.3.3 Determine Training Requirements to Drop 
for Cancelled Ratings/NEC 

1.2.4 Develop Requirements Due to Personnel 

Attrition 33 

1.2.4.1 Receive and Evaluate Information on 
Personnel Needs 

1.2.4.2 Develop Quantitative School Data and 
Quotas 



Vl-26 



TAEG REPORT NO. 12.-1 



ft «~ # 



y 

U4 

7 ^ T 
O *-i 
O ft T 

r> y 4 
« <v 
u u u 



I sA 
I I V) 
\A * 

o u * 

N- UJ 
>- V* 

I V CO 

I UNO 

i uj y y 

4 « M. < 

I 



a* £ 

UJ * ? 

N J w 
O * * ft. * 
• i -I m 

"> I < O < 

I » 7 et 

U. I < 4 

UJ i 

< ft Mi — _ « 

A 



/ 



/ 



/ 



< ♦ 



n l/> i 

U. W I 

— ( 

k/» » u. I 

>• ^ O f 

r x t* \ 

Ui UJ H I 

t H lij I 

ci ui i/i iy i 

O * >• i 

J M |/> » | 

'.O *J 1/1 I 

> Q S Q I 

UJ uj uj O | 

6*71 I 





1 ft 


<•> 




\ 1 


•* 




r> Z 






t O O 
1 u> 






<x cc 






o 






i a u. 


















UJ 


Z 


• 


* T 


n* 




UJ UJ 


o 




O £ 





UJ 

ft 
«-* 

r> 

Cf 

UJ 

«r 

o 
z 



Of 



n 
-j 

UJ 

> 

UJ 

i 

o 



7S" 



• 






1 T 


ft 






\ r> 




















7. 








LU 








o r 








ft 




4 M UJ 




o — v 




* > 


o • 


J7- 


• 


* ar" i/i » 


• * 


tU "> 


ft 


r C5 vl 




> « O 


ft 


IU.HO 


• 


U> «" iLf 


ft 


1 ?Wfl 


U. ft 


O ctf 


• 


i — r r 


ft 




• 




«Y » 


» # • • 


ft 





• « 



JZ J 

UJ 2 4 

O * 
O 

z * *-t 

O uj ^ 

w« ft. « 

< O UJ ✓> 

* *- ft. UJ 

I «■ O O 

I o <j y 

( u. t a * 

I Z m 7" X 

I n >• 4 a 



i 
i 

i 

i 
l 



9 

ERIC 



TAEG REPORT NO. 12-1 



1.2.5 Develop Requirements Due to Changes in 

Recruit Profile Input 40 

1.2.5.1 Receive Notice of Changes 1n Recruit 
Profile Input 

1.2.5.2 Observe Changes in Recruit Profile 
Input 

1.2.5.3 Develop Requirements for New Courses 
or Programs 

1.2.5.4 Develop Requirements for New Techniques 

Functional Description of NETS - Function 1.0 
Develop Training Requirements 

A general description of this function was presented in the 
outline and top-level flow of the preceding section. Further 
detail is shown in this section through a series of flows 
which expand key subfunctions. In addition to notes within 
the flows that elaborate on significant points, a general 
discussion of the flow is contained on the facing page. 
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d. Summary of Major Technological Gaps and Problem Areas - 
Function 1.0 Oevelop Training Requirements 

1) The training system does not participate sufficiently 
early In developing training requirements for new or 
modified systems/equipment. 

2) Requirements for ongoing training are not well defined 
for the most part. 

e. Idealized Approaches - Function 1.0 Oevelop Training Re- 
quirements 

The clear definition of training requirements is an absolute 
must 1n an Idealized training system. In the current approach 
much of the initial definition 1s performed well before the 
training establishment enters Into full participation. This 
problem 1s discussed in the context of Function 3.0 (e,), 
since analysis 1s needed to define training requirements. 
The type of analysis needed for this definition may be con- 
sidered as pre-analysls, 1n order to differentiate 1t from 
the later more detailed analysis. 

Pre-analysis yields the broad requirements of what the train- 
ing is for, the kinds of equipment or systems Involved, the 
expected length of training, number of personnel Involved, 
general training objectives, and all other pertinent Informa- 
tion to give a full definition of training requirements. 

Analysis gives much more detailed information which initially 
concentrates on the details of the position/job being trained 
for. This start serves as the basis for further analysis 
which defines what must be covered in the course. 

The training system must become involved early 1n the develop- 
ment cycle to participate in the definition of requirements 
for training. This will ordinarily require some instruction 
with contractors for hardware developments. All such 
relationships should be through the SYSCOM which has major 
development responsibility. 

Similarly, training needs which come up other than for new 
equipments must be defined carefully. Such needs may come 
from the Fleet, CNO, or higher authority. For example, assume 
a need is given for a particular kind of skill training. This 
may be as specific as "how to repair a particular piece of 
equipment" or "the operation of the 3-M system." It may be 
broader such as "first-aid" or "deck-seamanship." It may be 
very broad such as training for Shore-Patrol duty. 
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Such Inputs are not training requirements, but merely 
statements of perceived needs. In order to turn the need 
Into a requirement, careful pre-analysls must be performed. 
The steps of the pre-analysls are shown 1n Figure VI 1.2.6. 
Notice that part of this preanalysis consists of an evalua- 
tion of possible design deficiencies and the sending of feed- 
back Information to schools, as appropriate. The output 1s 
a clearly defined training requirement. This has emerged as 
a result of interaction with the originator of the need. 

Another type of training requirement 1s the determination of 
the number of people to be trained. This determination for 
"A" schools is done currently by means of a computer program. 
This 1s an effective approach and appears to have all the 
required features of an idealized system. However, the 
determination of number of students to be trained for "C" 
schools requires much improvement to achieve a condition near 
the Ideal. At a minimum, a validation of the conceptual 
approach and a computerization must be developed. These 
concepts are expanded in the discussion of Function 6.0, 
Figures VI 6.5.1 and VI 6.5.1.3. 
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Function 2.0 Coordinate and Control Training 

a. Definition 

Training coordination and control incorporates the primary 
management/supervisory subfunctions . These include the 
activities of headquarters/staff groups at all levels. 
While the subfunctions are the same for all levels, the 
degree of depth will be different, as is to be expected. 

The general command subfunctions which would pertain to any 
type of military organization are not included. Those that 
are included are the subfunctions pertaining most directly 
to training. 

Some subfunctions in other functional areas also deal with 
management/control. To a small extent there 1s an overlap. 
However, the primary distinction is that this section 
emphasizes coordination over two or more organizations that 
do the same thing. Where specific control subfunctions are 
called out, these tend to deal with directing operations for 
a single organization. 

b. Description 

The Coordinate and Control Training function is described 
through an outline which contains all of the functional 
elements presented in the functional flows, and in addition 
through an overall functional flow, Figure VI 2.0 - Coordinate 
and Control Training. 

2.0 Coordinate and Control Training .......... 49 

2.1 Develop Training Policy and Constraints 52 

2.1.1 Determine Heed for New Training Policy 

2.1.2 Review Policies and Instructions 

2.1.3 Maintain Knowledge of Technological 
Advances 

2.1.4 Draft Specific Policy for Subordinate 
Commands 

2.1.5 Develop Training Approach 

2.1.6 Develop Draft Instruction 

2.1.7 Distribute Draft Policy for Comment 

2.1.8 Incorporate Applicable Comments 

2.1.9 Drop the Approach 

2.1.10 Present Revised Draft for Approval 

2.1.11 Publish and Disseminate New Instruction 
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2.2 Monitor Activities 

2.2.1 Review Record of Milestones for 
Significant Actions 

2.2.2 Notify Subordinate Command 

2.2.3 Determine Reason for the Delay 

2.2.4 Assist as Required 

2.2.5 Set New Due Oate 

2.2.6 Receive Outputs 

2.2.7 Review Outputs; 

2.2.8 Prepare Comments and Return to Originator 

2.2.9 Receive Corrected Version 

2.2.10 Approve and/or Utilize as Needed 

2.3 Inspect Training Facilities/Operations .... 

2.3.1 Prepare for Training Inspection 

2.3.2 Make Administrative Arrangements 

2.3.3 Develop Inspection Plan/Procedure 

2.3.4 Conduct Training Inspection 

2.3.5 Collate and Organize Data 

2.3.6 Develop Findings and Recommended Actions 

2.3.7 Prepare Preliminary Statements for Report 

2.3.8 Conduct Debriefing 

2.4 Keep Aware of Resource Availability 

2.4.1 Develop Source/File of Resources 

2.4.2 Keep Running Account of Gains/Losses and 
Expenditures 

2.4.3 For Unprogrammed Requirements: Determine 
if Resources are Adequate 

2.5 Assign Responsibiliti ties/Task Subcommands . . 

2.5.1 Evaluate Nature of the Requirement 

2.5.2 Develop Guidelines for an Approach 

2.5.3 Determine Organization Best Qualified for 
the Task 

2.5.4 Task the Organization 

2.5.5 Develop Milestones 

2.5.6 Direct the Organization to Reallocate 
Resources 

2.5.7 Delay the Requirement 

2.6 Coordinate Activities 

2.6.1 Determine Best Methods of Dividing the Task 

2.6.2 Determine Specific Assignments 

2.6.3 Determine Best Methods of Fulfilling the 
Requirement 

2.6.4 Task Each Organization to Perform the 
Approach 

2.6.5 Obtain Outputs 

2.6.6 Chanqe Each to Reflect a Common Approach 

2.6.7 Assign Responsibility to One Organization 

2.6.8 Assign Assist Role to the Other Organiza- 
tion's) 

2.6.9 Obtain Integrated Output 

2.7 Develop Reports 

Request Inputs 

Develop Total/Composite Reports 
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c. Functional Description of NETS - Function 2.0 
Coordinate and Control Training 

A general description of this function was presented in the 
outline and top-level flow of the preceding section. 
Further detail is shown in this section through a series of 
flows which expand key subfunctions. In addition to notes 
within the flows that elaborate on, significant points, a 
general discussion of the flow is contained on the facing 
page. 
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d. Summary of Major Technological Gaps and Problem Areas - 
Function 2.0 Coordinate and Control Training 

1) There is no integrated training/education record for 
all personnel. 

2) The're is no clear check on instances where personnel 
are assigned outside of their area of specialized 
training. 

3) Whenever a new idea or policy is issued, it comes as an 
instruction. As a rule, there is not sufficient detail 
to tell commands, "how to do it." Consequently, there is 
a variety of interpretation, much of which is wide of the 
mark. 

4) There is generally inadequate communication between loca- 
tions. Excellent ideas at one place are unknown at other 
locations. 

5) There tends to be too much disparity between different 
locations teaching the same subject matter. 

6) Often, directives are issued on a priority basis, but no 
additional resources are allotted. Some other program 
has to give, but there is no adequate method of evaluating 
priority of programs. 

7} There is a singular Uck of standardization of terms for 
"places" where training occurs. 

e. Idealized Approaches - Function 2.0 Coordinate and Control 
Training 

Idealized approaches for this function consist of those general 
concepts and approaches to management which would offer greater 
effectiveness for the resources* expended in the 1980 time 
frame . 

A fundamental requirement is the establishment of a computerized 
training/education record for each officer and enlisted person 
in the Navy. This record must be easily updated, and rapidly 
retrieved, even from remote locations. Data to be entered would 
be all pertinent training and education "completions" including 
Onboard Training (OBT) and significant sections of Personnel 
Qualification Standard (PQS). The development of this kind of 
data base should be in conjunction with other corrmands such as 
tne Bureau of Naval Personnel, since the training/education 
record is really one facet of a total personnel record. 
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The current expansion of the Enlisted Master Tape is a 
start toward this idealized goal. Other trends pointing 
to a more effective data base are the proposed Navy Campus 
for Achievement-Automated Data System (NCFA-ADS), and the 
Fleet Ballistic Missile (FBM) personnel data bank. Work on 
the development of the FBM system has begun. This system is 
a model of what the future personnel/training/education 
record may be. 

A potential need for the idealized system is safeguards to 
assure that the Navy has full utilization of trained personnel. 
Essentially, this means that every effort should be made to 
assign personnel to billets that are commensurate with their 
formal training. In no way, should the prerogative of a 
commanding officer be interfered with. However, the over-all 
good of the total Navy must be considered. If an assignment 
outside of a trained skill is deemed necessary, such assign- 
ment must be made. However> such information should be con- 
veyed to higher headquarters so a decision may be made as to 
whether or not a more appropriately trained person might be 
sent to fill that particular billet. 

The potential loss to the Navy as a result of improper utiliza- 
tion of training is sizable. As an illustration, consider 
the fact that approximately 50,000 students complete "A" school 
per year; the average school length is 12 weeks, and the cost 
of training is $200 per week. If even only 1% of these 
graduates are not appropriately assigned (500 people), a total 
of $1,200,000 training dollars have been wasted. This does 
not consider additional loss of morale or possible negative 
influence on shipping over. 

There is a further requirement of an ideal system that the 
methodology of carrying out relatively complex techniques be 
specified clearly to all concerned. When an instruction is 
issued that requires new, unusual, or complicated methodology , 
there should be meetings/seminars/classes to supplement the 
written instructions and clarify implementation features. The 
conduct of such meetings gives the training organizations re- 
quired information, and at the same time conveys to the command 
issuing the instruction, a full understanding of implementation 
problems. 

Communication between locations must be improved for the ideal 
circumstance. At present, there is a fair amount of such 
interchange, but this must be ai'gme.ited considerably. While 
meetings and task forces are helpful, professional type con- 
ferences offer considerably more advantage. Such conferences 
would offer the presentation of "papers" dealing with 
significant common problems and innovative approaches. 
In addition, there would be much more opportunity to discuss, 
informally, issues of mutual interest. 
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The ideal system must have no significant difference between 
courses taught at different locations. Such differences will 
be minimized as individualized instruction is increased. 
However, even within that context, there may be courses or 
parts of courses utilizing conventional Instruction. 

To assure uniformity there should be central control of the 
development of curricula, programmed texts, guides, and hand- 
outs. This will assure that schools at different locations 
are teaching v.he same things in the same way using the best 
possible approach. 

The most effective kind of control would be by centrally 
developing such instructional items. The desired approach 
would be to use task forces of professionals and technically 
qualified personnel from the different training units that 
will use the "to-be-developed" curriculum. These would operate; 
under the direct guidance and control of a highly qualified 
professional nucleus. 

Behavioral objectives for all courses should be developed, 
reviewed, and revised in a similar matter. To an extent, 
the objectives are the total key to effective training. If 
a valid set of objectives are met, the trained individual 
will be able to perform his job assignment appropriately. 
This has to be the case since objectives must be based on a 
task analysis which reflects the job. 

Another requirement for idealization is a viable method for 
determining priorities. Often there is a need to choose be- 
tween programs, approaches, types of courses, etc. Without 
an effective priority selection, decisions may be questionable. 
The approach to computer simulation mentioned in the discus- 
sion of Function 3.0 (e.) is applicable also for considering 
priorities. The problem of priorities is discussed also in 
the context of Function 6.0. 

Another facet of an ideal system is consistency and uniformity 
of designating an instructional organization. The large 
difference in the size of training: centers, schools, detach- 
ments, units presents a most confusing picture. The origin of 
such differences is understandable since the present more 
centralized training structure picked up a number of relatively 
disparate elements. However, there is a need for a realignment 
of current terminology and organizations. Since organiza- 
tional structure is not part of the mission of this study, 
no further comment will be made on that point. However, the 
related terminology is covered in the discussion of Function 
4.0 (e.). 
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Function 3.0 Analyze and Plan Training 

a. Definition 

The Analyze and Plan Training function involves the develop- 
ment of concepts for training, the planning of how the train- 
ing will be performed, and the development of requirements to 
be input in order to determine the costs of implementation. 

For new or modified systems, the sequences depicted fit very 
closely. In such instances, the function fits into the re- 
quirements of OPNAVINST 1500. 8G ''Preparation and Implementa- 
tion of Navy Training Plans for New Developments," 28 June 
1972. However, this instruction 1s also Involved with the 
function of developing training requirements {Function 1.0). 

The function of training analysis and planning applies to on- 
going training primarily through continual evaluation of train- 
ing operations. Such operations include scheduling, analytic 
requirements, and resource requirements. 

Subfunction 3.4 - "Analyze Performance Requirements" is 
probably the most significant of the subfunctions. This deals 
with specific task analytic approaches which are the keys to 
optimal cost-effectiveness of training. 

b. Description 

The Analyze and Plan Training function is described through an 
outline which contains all of the functional elements presented 
in the functional flows, and in addition through an overall 
functional flow, Figure VI 3.0 Analyze and Plan Training. 

Page 
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3.1 Develop Training Concept 80 

3.1.1 Determine Maintenance/Operational Concept 

3.1.2 Determine Extent of Training Needed 
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3.4.2 Develop Personnel Qualification 

Standards (PQS) 94 

3.4.2.1 Obtain References for the Equipment/Job 

3.4.2.2 Obtain Skilled Personnel (Experienced 
on the Equipment/Job) 

3.4.2.3 Develop Tasks Within Major PQS Categories 
(of the PQS Outline) 

3.4.2.4 Develop Subtasks for Each Task 

3.4.2.5 Determine Level of Achievement for Subtasks 

3.4.2.6 Arrange in Logical Sequence 

3.4.2.7 Apply PQS Numbering System 

3.4.2.8 Develop Reference Information 

3.4.2.9 Develop PQS Card 

3.4.2.10 Review/Approve 

3.4.2.11 Publish/Catalog fc 

3.4.3 Develop Personnel Performance Profiles (PPP). . 96 

3.4.3.1 Collect References and/or System Data 

3.4.3.2 Obtain Technical Experts 

3.4.3.3 Evaluate Technical Data to Determine Skill 
Requirements (Knowledge and Skill Required 
to Operate and Maintain a System* Sub- 
system, or Equipment) 

3.4.3.4 List and Number in Accordance with 
WS-13581 (1 Jan 1971) 

3.5 Develop Training Plan 98 

3.5.1 Determine Type and Number of Courses 

3.5.2 Determine Student Loads and Quotas 

Prerequisi tes 

Number to be Trained (by Class and Course) 
Available Facilities and Training Equipment 
Number That Can be Accommodated - by Class 

3.5.3 Determine Instructor/Staff Requirements 

Number 

Characteristics 
Training Required 

3.5.4 Determine Requirements for Training Support 

Facilities 
Training Literature 
Supplies 

Training Materials 

3.5.5 Develop final Schedule 

Acquisition of Resources 
Preparation for Training 
Training Implementation 

3.5.6 Revise Schedule as Appropriate 
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Functional Description of NETS - Function 3.0 
Analyze and Plan Training 

A general description of this function was presented in 
the outline and top-level flow of the preceding section. 
Further detail is shown in this section through a series 
of flows which expand key subfunctions. In addition to 
notes within the flows that elaborate on significant 
points, a general discussion of the flow is contained on 
the facing page. 
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Summary of Major Technological Gaps and Problem Areas - 
Function 3.0 Analyze and Plan Training 

1) The training establishment 1s not involved during the 
early (and crucial) stages of developing the training 
concept for new equipment or major modifications. 

2) The planning process for training requires an update 1n 
methods to improve the effectiveness of training plans. 

3) The modified task analysis, which 1s used predominantly, 
is a stop-gap measure. 

4) The task analytic and PQS approaches seem to be on parallel 
paths. While small differences are apparent, the outputs 
are clearly similar. 

5) The "Qualifications for Advancement in Rating" (NAVPERS 
18068C) should be 1n sync with Task Analys1s/PQS. There 
is an apparent separation between what the Individual 
does on his job and what he must do/know 1n order to 
advance. 

Idealized Approaches - Function 3.0 Analyze and Plan Training 

Idealized approaches to this function Involve those operations 
which optimize the analysis and planning procedures. Analysis 
involves the determination of the who, when, where, and what 
of training. Such analysis for new systems or modifications 
begins within the development cycle, and tends to be performed, 
largely, by the contractor. In the life cycle of a system 
there are four distinct phases: 

I Conceptual 

II Validation 

III Full Scale Development 

IV Production and Operational 

Currently, there is no active involvement by the training 
system until the third phase — Full Scale Development. How- 
ever, significant events occur in the earlier two phases. 
These are shown in Figures VI 3.6 and VI 3.7. Those events 
relating directly to training operations are shown 1n blocks 
with accentuated black lines. It is evident that many 
significant training decisions are made early in the develop- 
ment cycle. For example, as described in NAVSHIPS 0900 060-0380: 

"The Personnel and Train ing Plan describes the training 
required to achieve the skill levels set forth in the 
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(From NAVSHIPS 0900-060-0380) 
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determination of ship's manning. First, the 
skill requirements are defined 1n detail, assign- 
ing rates and NEC codes to personnel. Next, the 
formal training course requirements are identified , 
describing for each course the element? of the course 
depicted in [Figure VI 3.7]. The team training re - 
quirements for combat team training, and for propul- 
sion team training ashore 1n recent snip develop- 
ments, are Identified. Of special significance to 
ship system design 1s the description of simulation 
requirements for conducting on board combat team 
training, in that hardware and software will normally 
be required for the specific purpose of simulating 
combat conditions." 

Such outputs are largely the result of contractor analysis 1n 
fulfilling the data requirements of a hardware development. 
The training establishment does not get Involved to any extent 
until the Naval Training Plans Conference (NTPC) is called. 
However, the NTPC 1s not held until the Validation Phase 1s 
completed. 

It is necessary that representatives of the training system 
become Involved much earlier than the NTPC. Such inter- 
action should include early relationships with the Inte- 
grated Logistic Support (ILS) activities of the contractor. 
The closest connection should be in the t*sk analytic effort 
and in the definition of needed personnel, training, and 
training equipment. In the earliest stages, liaison and 
coiiitient on outputs could be sufficient. As the development 
proceeds, the Involvement may become deeper. Of course all 
such interaction must oe through the SYSCOM that has develop- 
ment responsibility. 

Such early involvement will allow a longer planning and prepa- 
ration cycle for training. Also, it will assure the develop- 
ment of realistic training plans in a much more effective 
manner than the current approach. This is a particular need 
in the case of relatively long lead items such as simulators 
or complex trainers. 

The idealized system requires optimal planning. Aspects of 
"optimum" include: uniformity in planning across organiza- 
tions, use of equal cost/usage and other factors, capability 
to include contingencies, and the capability to project Into 
a future time period. 

An initial requirement is a set of common planning factors 
for use by all elements of the NETS. Such factors must be 
centrally developed, maintained, and controlled. In addition 
uniform procedures for the utilization of these factors must 
be developed and made mandatory for all organizations. 
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Figure VI 3.7 - Validation Phase (Phase II) of Ship Life Cycle 
(From NAVSHIPS 0900-060-0380) 
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Factors should, as applicable, be expressed in terms of 
ranges or sets with associated assumptions to permit the 
planner to develop alternatives or to revise plans. Alter- 
natives, as well as projections, further require the usage 
of mathematical models and computer simulation. The use of 
such management tools would allow the projection of various 
kinds of situations which may oe encountered in the futur , 
or as a result of unforeseen or unpredictable events. As 
an example, contingencies may be planned in advance for: 

increase in student load 
decrease in student load 
increase in course length 
decrease in course length 
change in staff strength 
change in available resources. 

By means of computer simulation, based upon an appropriate 
mathematical model, the expected effects of alternative 
approaches may be spelled out closely. The lack of such 
capability currently is a serious handicap to effective 
planning. This problem must be met for the future idealized 
system. 

The idealized planning approach would always contain viable 
alternatives, each with a well defined resource plan and 
assumptional baseline. 

Decisions made under those conditions would allow the evalua- 
tion of specifically defined assumptions. Since monitoring 
would include the determination of change in the assumptions, 
the process of controlling the implementation of such plans 
would become much easier. 

In addition the use of computer simulation, together with 
appropriate factors, would allow full consideration of the 
cost-effectiveness for a wider range of alternatives in the 
area of training technology, i.e., classroom instruction vs 
on-the-job instruction, full task trainers vs part task 
trainers, film vs slides, etc., in order to achieve an optimal 
plan. 

Another required characteristic of training plans for the 
idealized system is that the plan must spell out the means for 
expediting the fast learner and for helping the slow learner. 
Such strategy and tactics are based on the assumption that 
training is largely, if not completely, individualized. It 
is conceivable that ultimately such strategies may become 
standard operating procedures. But, until this happens, all 
planning must include these specifics. 
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The Ideal 1 zc^ approach to training is totally dependent 
upon an effective task analysis. The current approach to 
task analysis in the NETS is the modified task analysis. 
This 1s largely a type of task inventory , since the details 
of task? to be performed are not specified. While this 
technique is a vast advance over what had been done pre- 
viously, it cannot be considered o be ideal. 

The ideal method requires the determination of which tasks/ 
subtasks/actions must be performed and the development of 
standards of performance. Such determination must be done 
by a combination of methods such as consulting with the 
system designers; evaluating the maintenance engineering 
analytic data, reliability data, Operational Sequence 
Diagrams (OSD), etc. For new or modified systems/equipment > 
there is no other data. For hardware/systems in active use, 
there is additional information. Under such circumstances, 
the initial documentation should be used as the basis for 
interviews with operating personnel. Such interviews should 
determine: the extent to which all the required operations 
are performed, those actions which are performed that are not 
in the "required" category, problems encountered, frequency 
of operations, etc. Interviews should be on a sampling basis, 
but the sample should be developed to be representative, 
rather than random. The final output would be those tasks, 
subtasks, and actions which must be performed; the standards 
of achievement required; the references/manuals needed; the 
tools and support equipment needed; and any other pertinent 
information required to delineate the job under study. This 
kind of product essentially negates the need for a separate 
"training task analysis," since the details provide a direct 
basis for training. 

Once the task analysis is completed, it may be updated as the 
result of feedback data, including operational and failure 
analysis from such approaches as the 3-M system, NAVSEC data, 
etc. Of course if a major modification occurs, the total 
analysis should be revalidated. 

The methodology described is a rigorous, demanding one which 
requires the use of trained and experienced professional 
personnel. While some "subject matter" experts (operational 
personnel) should be involved as aids, they cannot perform 
the required analysis effectively. The time factor for such 
task analysis would not be significantly longer than for the 
technique used currently. The output would be more effec- 
tive. 
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An approach such as the PQS technique clearly belongs to 
an idealized system. However, the duplication of analytic 
effort does not fit. A single Integrated data base must be 
utilized. For the most part, the task analysis method 
described previously will suffice for PQS. If some parts of 
PQS should require extension or elaboration, this may be 
done readily as an Integral part of the analysis. 

This same base of analytic data should be the foundation for 
the advancement in rate examinations. Currently, a separate 
analysis 1s made, apparently, for the qualifications upon 
which the examinations are based. The idealized situation 
must not have this multi -track mode. The integrated task 
analysis approach would end the current separation between 
what the individual does on his job and what he must do and 
know 1n order to advance 1n rate. In the light of expected 
future social pressures to employ increased numbers of 
culturally deprived personnel, it is important to remove the 
possible Impression that the examinations are artificial 
barriers to advancement. Making the job and the requirements 
for advancement in consonance will rectify this situation. 
In addition a rapprochement in analytic approaches 1s needed 
1n order to save manpower and 1n order to prevent the possi- 
bility of disparate outputs. 
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Function 4.0 Implement Training 
a. Definition 



The Implement Training function covers all activities which 
are directly concerned with the instruction of students. 
Included is the preparation and evaluation of: instructional 
personnel, administrators, and curricula; as well as the 
actual operations of conducting and administering training. 

The primary inputs to this function are training plans and 
task analytic data. The flows cover both the development 
of a new course and course revisions. For revisions, only 
parts of the sequence may apply. However, review of courses 
to determine if revision 1s needed should touch each of the 
subf unctions and tasks. 



b. Description 

The Implement Training function is described through an out- 
line which contains all of the functional elements presented 
in the functional flows, and in addition throuqh an overall 
functional flow, Figure VI 4.0 - Implement Traininq. 



Page 

4.0 Implement Training Ill 

4.1 Develop Curriculum Outline 114 

4.1.1 Determine Topics to be Included 116 

4.1.1.1 Evaluate List of Potential Topics 

4.1.1.2 Eliminate Duplications, Consolidate 
Overlaps 

4.1.1.3 Organize Topics in Logical Sequences 

4.1.2 Determine Sequence of Topics 118 

4.1.2.1 Analyze Topic Relationships 

4.1.2.2 Determine Logical Order of Presentation 

4.1.2.3 Arrange Topics in Logical Sequence 

4.1.2.4 Eliminate Redundancy" 

4.1.2.5 Revise Sequence 

4.1.2.6 Review Topic Sequence 

4.1.3 Determine Specific Objectives 120 

4.1.3.1 Analyze Each Topic 

4.1.3.2 Determine Logical Objective(s) for Each 

4.1.3.3 Evaluate Initial List of Objectives 

4.1.3.4 Eliminate Redundancies/Overlaps 

4.1.3.5 Arrange Objectives in Sequence by Topic 

4.1.3.6 Re-evaluate New Listing of Objectives 

4.1.3.7 Revise Objectives and/or Sequence 
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4.1.4 Determine Method(s) of Presentation 122 

4.1.4.1 Evaluate Potential Methods 124 

4.1.4.1.1 Determine Advantages/Disadvantages 
of Each Method 

4.1.4.1.2 Determine Resources Needed for 
Each Method 

4.1.4.1.3 Estimate Lead Time for Each Method 

4.1.4.2 Evaluate Topics/Objectives for Best 
Presentation Method(s) 

4.1.4.3 Determine if Some Method(s) is/are 
Precluded 126 

4.1.4.3.1 Determine Policy on Methods 

4.1.4.3.2 Determine if a Method is Inapplicable 

4.1.4.3.3 Determine if a Method is Particularly 
Suitable 

4.1.4.3.4 Eliminate Method From Consideration 

4.1.4.4 Designate Each Topic With Method(s) 
to be Used 

4.1.5 Determine Method (s) of Evaluation 128 

4.1.5.1 Consider Potential Methods of Evaluation 

4.1.5.2 Evaluate Best Method(s) of Testing for 
Each Objective 

4.1.5.3 Select Evaluation Method(s) for Each 
Objective 

4.2 Develop Course Details .... 130 

4.2.1 Develop Instructor Guide 132 

4.2.1.1 Determine Points to be Covered for 
Each Topic 

4.2.1.2 Determine Method(s) for Presenting Points 
1n Each Topic 

4.2.1.3 Determine Support Material (s) and Train- 
ing Aids Required 

4.2.1.4 Determine Time per Topic 

4.2.1.5 Determine Test Methods 

4.2.2 Develop Lesson Plans 

4.2.3 Develop Student Guides 

4.2.4 Develop Requirements for Training Aids ... 134 

4.2.4.1 Select Topics/Points for Elaboration/ 
Augmentation 

4.2.4.2 Determine Type of Aid(s) 

4.2.4.3 Develop Descriptive Details 

4.2.4.4 Develop the Aid(s) 

4.2.4.5 Send Details to Support Center 

4.2.4.6 Evaluate Completed Aid(s) 

4.2.4.7 Revise as Required 

4.2.5 Develop Programmed Instruction 136 

4.2.5.1 Analyze for Feasibility of Application 

4.2.5.2 Recheck Objectives and Revise as 
Needed 
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4.2.5.3 Write Objectives (if Course is New) 

4.2.5.4 Develop Detailed Outline 

4.2.5.5 Develop Frame I terns 

4.2.5.6 Develop Criterion Tests 

4.2.5.7 Integrate into Trial Series 
4.2.5.6 Administer to Group for Test 

4.2.5.9 Evaluate Test Output 

4.2.5.10 Revise as Needed 

4.2.5.11 Submit for Approval 

4.2.5.12 Submit for Publication 
4.2.6 Develop Testing Approaches 

4.3 Develop Personnel 138 

4.3.1 Develop Administrators 140 

4.3.1.1 Obtain School Training in 
Administration 

4.3.1.2 Obtain On-the-Job Training 

4.3.1.3 Obtain Job Experience in Administration 

4.3.1.4 Receive Evaluation and Guidance 

4.3.1.5 Receive Additional Administrative Training 

4.3.1.6 Receive Reassinnment 

4.3.2 Develop Instructors 142 

4.3.2.1 Attend Instructor Training Course 

4.3.2.2 Obtain Job Experience as an Instructor 

4.3.2.3 Obtain Job Experience as an Instructor 
"Aide" 

4.3.2.4 Receive Additional Training as an 
Observer/Auditor in Class 

4.3.2.5 Evaluate Instructors 

4.3.3 Develop Learning Supervisors 

4.3.4 Develop Programmed Instruction Course Writers 

4.3.5 Develop Education Specialists 

4.4 Conduct Training 144 

4.4.1 Orient Class 

4.4.2 Present Overview of the Course 

4.4.3 Distribute Initial Handouts 

4.4.4 Describe Approaches to be Taken 

4.4.5 Conduct Class Sessions 

4.4.6 Monitor, Appraise, Counsel 146 

4.4.6.1 Keep Record of Student Progress 

4.4.6.2 Maintain Informal Checks With Students 

4.4.6.3 Look for Signs of Falling Behind 

4.4.6.4 Determine Potential Slow Students 

4.4.6.5 Determine Individual Problem Areas 

4.4.6.6 Evaluate Course Content 

4.4.6.7 Determine Potential Training/Performance 
Aids 
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4.4.6.8 Keep Available for Counseling - at 
Student's Initiative 

4.4.6.9 Call In Students With Possible Problems 

4.4.6.10 Explore the Nature of the Problem 

4.4.6.11 Aid Student in Developing Alternative 
Approaches 

4.4.6.12 Suggest Other Options 

4.4.6.13 Terminate Counseling Session 

4.4.6.14 Follow-Up 

4.4.7 Complete Class Presentations 

4.4.8 Evaluate Course (Conduct and Content) 



4.5 Evaluate Training (Testing) 148 

(Conventional Te sts' 

4.5.1 Develop Test Outline(s) 

4.5.2 Develop Test Items 

4.5.3 Administer Tests 150 



4.5.3.1 Develop Scoring Method 

4. 5. 3. 2 Develop Test Instructions 

4.5.3.3 Develop Secure Test Procedures 

4.5.3.4 Administer the Test 

4.5.3.5 Grade the Test 

4.5.3.6 Analyze Test Data 

4.5.4 Evaluate Items (Item Analysis) 152 

4.5.4.1 Record Item Data 

4.5.4.2 Calculate Percent Wrong for Each Choice 

4.5.4.3 Calculate Discrimination Index 

4.5.4.4 Develop Item Selection Criteria - Cut- 
Off Point 

4.5.4.5 Apply Selection Criteria 

4.5.4.6 Revise Items - Discard Poor Ones 
(Criter ion Tests) 

4.5.5 "Analyze Course Objectives 

4.5.6 Develop Specific Criteria for Each Objective 

4.5.7 Develop Standard(s) of Achievement 

4.0.8 Develop Evaluation Methods 154 

4.5.8.1 Develop Optimal Method of Determining 
Achievement 

4.5.8.2 Apply the Method 

4.5.8.3 Evaluate the Method 

4.5.8.4 Revise As Needed 

4.6 Administer Training 156 

4.6.1 Manage Resources 158 

4.6.1.1 Develop Cost Inputs 

4.6.1.2 Determine Operating Costs 

4.6.1.3 Determine Rates of Expenditure 

4.6.1.4 Develop Needs for Additional Resources 

4.6.2 Manage Students 160 

4.6.2.1 Enroll Students 

4.6.^.2 Administer Student Logistical Needs 
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4.6.2.3 Record Student Oata 

4.6.2.4 Discipline Students 

4.6.2.5 Set Back Students 

4.6.2.6 Dlsenroll Students 

4.6.2.7 Graduate Students 

4.6.3 Manage Operations 162 

4.6.3.1 Implement/Develop Policy 

4.6.3.2 Acquire Staff Personnel 

4.6.3.3 Train Staff Personnel 

4.6.3.4 Counsel Staff Personnel 

4.6.3.5 Evaluate Staff Personnel 

4.6.3.6 Reassign Staff Personnel 

4.6.4 Manage Data 164 
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Graphs 

4.6.4.3 Develop Required Data Inputs to Higher HQ 
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Functional Description of NETS - Function 4.0 Implement 
Training 

A general description of this function was presented in the 
outline and top-level flow of the preceding section. Further 
detail 1s shown in this section through a series of flows 
which expand key subfunctions. In addition to notes within 
the flows that elaborate on significant points, a general 
discussion of the flow is contained on the facing page. 
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Summary of Major Technological Gaps and Problem Areas - 
Function 4.0 Implement Training 

Task analytic data 1s not used as widely as possible. 

While the policy 1s to use Individualized Instruction 
wherever possible, the majority of the training 1s still 
lock-step. 

There 1s no analysis or on-going research to determine 
what kinds of Instruction can or cannot be Individualized. 

Where Individualized Instruction is used, 1t tends to be 
exclusively programmed text-linear program. 

The modular approach 1s not used as widely as is possible. 

There is a wide disparity 1n the effectiveness of curriculum 
development. 

Behavioral objectives and standards are not used as widely 
as 1s possible. 

Criterion tests are not used as widely as is possible. 

On-board training (OBT) 1s not utilized effectively. 

There 1s no feedback approach to the evaluation of PQS, 
used for OBT. 

Professional personnel and professional training/experience 
are in very short supply. 

No guidelines are apparent for evaluating the quality and 
outputs of staff personnel other than Instructors, par- 
ticularly at the school level. 

The evaluation of instructors lacks standardization. 

The attrition rate in schools places an extra burden on 
the training establishment. 

Development of training/performance aids is not sufficiently 
stressed. 

Greater emphasis upon reducing cost of operation is needed. 
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e. Idealized Approaches - Function 4.0 Implement Training 

Idealized approaches for this function are directly dependent 
upon the kinds of changes that occur 1n the Navy over the 
next ten years. It 1s assumed thfit a very desirable change 
will be to reduce, 1f not totally abolish occupational (across 
the board) training, and to substitute job specific training. 
Studies must be Instituted to determine the extent to which 
the "A" school concept <s cost-effective as compared to job 
oriented courses. A prime variable to consider 1s the fact 
that the ship-over rate for first term enlistees 1s less than 
twenty percent, and 1s even lower for some critical skills. 

Research 1n this problem may also evaluate, 1n a systematic 
manner, the degree of commonalty between equipment, systems, 
and different kinds of training. The objective would be to 
determine the maximal extent of modular training that may be 
developed. A simple computerization, for example, could search 
all naval subject areas 1n a complete manner. This could be an 
area of research with an Immense pay-off potential. A related 
search for commonalties should Include all service courses. 

It 1s further necessary to achieve a standard unit and terminol- 
ogy for training organizations. This should be standard, not 
only within the Navy, but across the other services. In de- 
scribing naval training It is essentially impossible to know 
how many "schools" there are since schools vary from organiza- 
tions that covar only one or two courses to those that involve 
dozens or more courses. In addition, training takes place 1n 
organizations calluj centers, detachments, RAGS, FRAMPS, etc. 
This variation 1s further confounded within the services. In 
the light of PL 92-436 and the report required by this law 
(Military Manpower Training Report) 1t is clear that standard 
approaches and terminology within and between the services will 
be eventually made mandatory by Congress through the Department 
of Defense. 

The NETS should consider a different type of training structure 
which would keep the highest training unit as a "school or 
center," but would change many of the present schools Into 
departments or sections. This approach fits into the concept 
of job specific training, since the school/center could be 
organized around groups of jobs. A further breakdown would be 
around course groupings. The elementary breakdown, after 
Recruit Training would be Basic Courses . These would pertain 
to the operation/maintenance of specific equipments. To the 
extent that modularized preparatory training is pertinent 1t 
would be utilized. For example the current Basic Electricity 
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and Electronics (BE/E) course end the developing Propulsion 
Engineering preparatory course would be applicable. In 
addition, 1t is expected that other developments fron the 
ILOG (San Diego) will be useful also. The Bask Course v:ould 
largely, 1f not totally replace the current "A 11 courses. The 
only "A" courses, or their future equivalents that should be 
retained are those that are demonstrably, through objective 
research study, cost-effective. 

The Basic Courses would be followed by Advanced Courses and 
Specialized Courses. Advanced Courses would*~present~more 
details of th~e job, but would emphasize subsystem and system 
aspects. Thus, such courses represent a logical career develon- 
ment path from "black boxes" (specific equipments) with very 
little subsystem involvement, to a greatly increased system 
capability. This further fits into the Combat System concept. 
In addition, all such courses must involve some basic principles 
of leadership, management, OBT supervision, and work supervision. 
Students will be primarily petty officers, whose jobs will re- 
quire the organization and work assignments of subordinates, as 
well as the active training of non-school trained personnel. 
This latter part of the course should be modularized and used 
for all Advanced Courses. 

Specialized Courses pertain to the training for major modifica- 
tions, speplal Requirements , refresher/update training; more 
complex* or advanced systems training, etc. To an extent, such 
courses would be equivalent to current "C" courses* However, 
as much as is possible, such courses should be replaced by self 
study packages (O^T). This should present no problem for non- 
equipment oriented courses. To an extent it may apply to 
other courses also. All such courses would involve criterion , 
tests which may be administered by ships officers. Full train- 
ing credit should be awarded after successful completion. 

It is necessary that the primary training mode be individualized. 
This will allow the most capable people to finish the course at 
a relatively rapid rate, and will further allow slower learners 
a better chance to complete the course successfully. 

A vitally necessary concomitant requirement is that the 
students receive assignment orders well before the shortest 
possible time to complete a course. If this is not done, the 
benefits of accelerated training are lost. In addition, this 
concept requires that a large proportion of the students in 
8asic Courses be assigned to specific stations/ships before re- 
porting to the school. This gives knowledge of the kind of 
equipment the student will work on after reporting to his duty 
station. Such information is needed for effective job training. 
It is assumed that prior assignments cannot be made in all 
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cases, since some flexibility must be maintained. Where 
there 1s not prior assignment, the number of people trained 
on different equipments would be based on a historical basis. 
All efforts would be made to assign a student appropriately. 

An essential part of the foregoing training concepts 1s 
effective OBT, since 1t 1s advocated 1n this approach that 
recruits with a 4-year or lower obligation go directly to 
the Fleet after recruit training. Training onboard may be 
performed through a combination of PQS, self study texts, 
programmed instruction, or any other potential means. Studv 
programs, behavioral criteria, and Informal criterion tests 
would be developed for each position requiring training. As 
mentioned before, petty officers would receive Instruction 1n 
conducting OBT as part of their Advanced Course. 

In addition, each lead petty officer would have direct respon- 
sibility for the training of his subordinates. This would be 
checked by setting up individual learning time-tables with 
milestones specifying when a striker has achieved certain 
behavioral criteria. This aspect of each petty officer's job 
would be one consideration 1n determining his potential 
recommendation for taking the advancement 1n rate examination. 
His effectiveness would also be judged 1n the evaluation of 
his Division Officer. If personnel are trained on-board 
effectively 1t must be through the leadership of his officer? 
and petty officers. If they are deficient 1n conducting/super- 
v1sing/mon1tor1ng, such lacks may well be considered in* develop- 
ing personnel evaluations (Fitness Reports). This Idea for 
motivating supervision of OBT is a type of manaqement by 
objectives. One of the essential objectives is real OBT. The*.* 
objectives, and the standards for their determination are 
specified. Failure to achieve such objectives must reflect on 
supervisory personnel. 

This Idealized approach 1s exemplified 1n Figure VI 4.7 Idealized 
Personnel Flow (1980 Time Frame). All Incoming personnel m'jst 
receive R ecruit Training . The length of this training must be a 
function of what 1s needed and how much can be learned aboard 
ship. Since the amount of training needed is currently largely 
a natter of opinion, 1t Is suggested that a research program 
be instituted to determine the necessary training. It may well 
be, also, that more capable personnel may be able to go through 
a shorter, albeit, more rigorous accelerated course as compared 
to average and slow trainees. 

After Recruit Training, the 6-year enlistees may go to a Basic 
Course , if qualified, or to the Fleet ( OBT). Those enlisting" 
for 4-years go directly to the Fleet (OBT) . This initial assign- 
ment to the Fleet 1s designated OBT, since a requirement will 
be that they train within a specific job context as described 
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previously. Job assignments for such personnel must be 
made through BUPERS, based upon ships needs. This must be 
done to control the number of people in various NEC's. 
Olrect assignments to the Fleet will not hurt recruiting, 
if the OBT concepts are followed. If anything, 1t may help, 
since many individuals are eager to get to sea rapidly. The 
point that 1s critical is that training will occur and be 
effective 1n direct relationship to the trainees' capabilities 
and motivation. 

The 6-year enlistee may go to an Advanced Course , Specialized 
Coyjse., or Fleet duty. For subsequent enlistments there would 
be alternation between Fleet/Shore duty and specialized train- 
ing as appropriate. A coveted assignment under this concept, 
for both officers and enlisted men, would be to a training 
unit, since this would help them perform the OBT responsi- 
bilities more effectively. 

The 4-year enlistees have an option of going to Basic, Advanced, 
or Specialized Courses if they extend their enlistments or 1f 
they ship-over. On the basis of current figures, it 1s ex- 
pected that approximately eighty percent of these Individuals 
will not stay In the Navy beyond the initial enlistment. On 
release, there will be a further option of entering the Naval 
Reserve or receiving a discharge. 

Standardization of OBT will be accomplished by the packages/ 
courses/guides supplied to the ships/stations by CHNAVTRASUPP. 
Preparation of these materials would be primarily by the schools, 
guided by CHNAVTRASUPP, Additional sources of development may 
be within CHNAVTRASUPP, or by contract. 

It 1s visualized that training units of the future will have 
sizeable staffs involved in the preparation and revision of OBT 
materials including training/performance aids. Also, very few 
conventional instructor personnel will be needed. Most of the 
instructional staff will consist of Learning Supervisors, 
Program Writers, Test Writers, and Counselors. Further effort 
will involve close support to the Fleet for training problems 
as well as a careful application of feedback approaches (see 
Function 5.0). 

The preceding exposition is an overall outline for a future 
instructional subsystem. Major interfaces are discussed and 
their significance has been stressed. Unless the Interface 
assumptions are met the basic tenets of tho future system 
cannot be fulfilled. 

The following section of this discussion will consist of brief 
descriptions of specific idealized situations. Some of these 
paragraphs will reinforce earlier statements, others will 
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supplement them. For the most part, these "Ideals 11 will be 
directed toward the summary of major technological gaps and 
problem areas presented in (d.). 

Specific idealized approaches: 

1) Task analytic data must be the basis for all curricula. 
Where earlier analysis has been utilized, complete and 
effective feedback must be obtained to validate and/or 
correct the curriculum. There must be an assumption that 
initial analysis may have defects, as well as the fact 
that the details of a job may be changed. 

2) Training should be by means of individualized Instruction 
as much as is possible. No instruction should be exemDt 
from "individualization 11 without excellent justification 
and the concurrence of upper command. However, non- 
concurrence carries the obligation of pointing out "how" 
to individualize the subject matter under discussion. In 
part, this ideal is based on the assumption that movement 
orders will be prepared well before the shortest time 
possible to complete a self-paced course. 

3) If individualized instruction does not seem to be applicabl 
in some instances, research should be instituted to 
determine the pertinent variables and if individualization 
might be approached in such a way as to make it feasible. 

4) Branching approaches to individualized programs should be 
used more extensively, since these allow the most capable 
students to advance much more rapidly than linear programs. 
Research programs should determine the extent of time 
differences for course completions in order to establish 
data pertinent to develop cost-effectiveness of branching 
vs linear approaches. 

One problem is the writing of branching programs. This is 
a much more difficult task than the linear approach. The 
feasibility of training enlisted personnel for the writing 
of such programs should be explored. If this is not a 
likely approach, a small nucleus of well qualified Educa- 
tional Specialists might be used. 

5) It is essential to extend the application of training 
commonalties and to develop training modules that are 
common for different specialties. A more effective method 
of finding such points of similarity is required. An 
immediate solution that emerges is the use of a computer 
program. If data for every topic of every major course 
were searched appropriately, commonalties could be 
established readily. 
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6) The envelopment cf curricula 1s a difficult and demanding 
task 1f performed correctly. To assure that these 
developments are maximally effective the preparation of 
curricula and course materials must be centrally con- 
trolled to a much greater extent than 1s the case at 
present. Virtually all programmed text materials, CAI 
programs, and CMI programs must be either centrally pro- 
duced or produced on a tasking basis by a force covering 
several different commands plus other services as appro- 
priate. Much of the other curricula material should be 
developed 1n the same manner. 

7) it 1s a fundamental requirement for effective training 
that the goals and objectives of the training be specified. 
Consequently, all courses must involve behavioral objectives. 
These should be carefully prepared and controlled since they 
form the foundation of the curriculum. 

8) The Ideal method of testing to determine if behavioral 
objectives have been achieved is by means of criterion 
tests and clear standards for evaluation. These must be 
Integral parts of all courses and should be centrally pre- 
pared and carefully controlled as a principle internal 
measure of quality control. The external index 1s "Feed- 
back." 

9) The amount of OJT must of necessity Increase 1n the light 
of these concepts. The Fleet will be getting some school 
people that are especially trained for a job as assigned. 
If they are assigned to different jobs, the ship must train 
them/ However, other new personnel will report that have 
not been job trained. The motivation on the ship must be 
to see that such personnel learn rapidly. Such "ship" 
motivation to encourage OJT 1s not apparent currently. 

10) An evaluation and feedback approach to PQS is essential. 

No Instance of revision of a PQS was found. It is unlikely 
that some changes are not needed. In addition, a study 
must be made of how PQS 1s used aboard ship - methods used 
and problems encountered. The objective of such a study 
would be to develop methods of more effectively using PQS. 
This also 1s a needed research area. 

11) The ideal situation requires professionally trained personnel. 
It is necessary to establish specific career fields for 

both officer and enlisted personnel. This would entail 
training for all phases - administrative and instructional, 
as well as a defined "growth ladder." 

Part of the career field must consist of Educational 
Specialists, to provide continuity of effort at training 
units. There must be definitive goals and objectives 
established for the development of these specialists. In 
addition, there should be safeguards against the misuse of 
such personnel . 
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12) There should be specific guidelines and objectives 
established for the administration/management of train- 
ing units. In addition, there should be standards for 
the evaluation of these functions. The ideal situation 
requires superior management. This requisite can be 
achieved by the use of the preceding tools. Application 
of such methods is, essentially, management by objectives. 

13) Judged over the wide spectrum, the evaluation of instructors 
requires improvement. The basic course gives him excellent 
fundamentals. These are crammed in, because of time re- 
strictions. When the new instructor starts, he is not 
fully prepared and he cannot be so considered until he has 
had at least two complete courses under careful supervision 
Such supervision must include close evaluation and monitor- 
ing. 

Three separate instances of technical deficiencies of in- 
structors were found, even though in no case were questions 
addressed to that point. This indicates that no assump- 
tion may be made that instructors are fully knowledgeable 
in their technical field. With increased individualized 
instruction, this problem will diminish, but until all 
instruction is individual, the instructor should be re- 
quired to study the subject matter and sit in on a course. 

14) The idealized situation is one in which the attrition rate 
is at the lowest possible point. Even 1f it were as low 

as ten percent, this means a sizable extra amount of trainee 
input is needed to maintain the required output. 

A drastic reduction in attrition might be achieved by 
handling trainees in a more effective manner than is now 
current. In cases that may need remedial aid, such as 
reading improvement, this must be available. The applica- 
tion of counseling must be extended. If the load warrants 
a full time counselor, the required number must be added to 
the staff. All staff personnel must receive increased 
training in counseling. All possible effort must be made 
to "save" a student, short of lowering standards. However, 
a cost-effective point of "no return" is needed also. 
Students should not be.,carried beyond that point unless the 
circumstances are unusual. 



There would also be a close^ analysis of the kinds of 
problems/situations/subject matter areas which tend to 
produce the greatest amount of academic losses. This is 
an important area for potential research, since the reasons 
given in the school reports of attrition are relatively 
incomplete and fragmentary. 
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One potential area of analysis and research 1s within 
the structure of the course Itself. For example, some 
parts of courses may be particular stumbling blocks. 
Perhaps these are basic arithmetic or mathematics, a 
physical principle such as Boyles Law, or a particular 
manipulative skill such as splicing a wire cable. If 
behavioral objectives form part of the course, as they 
should and will 1n the idealized situation, the analysis 
1s simplified as shown in Figure VI 4.8. It shows the 
need to determine which objectives are problem areas. 
Once these are highlighted, decisions must be made as to 
whether the objectives are valid or are really needed. 
A valid, but unneeded objective might be learned more 
effectively by means of OBT. If the objective must remain, 
the next decisions Involve whether or not the presentation 
can be Improved, or if another approach might be taken. If 
changes are possible, they should be made. If such change 
does not appear to be a possibility, a clear requirement' 
for research is evident and should be developed appro- 
priately. 

Without clear behavioral objectives, the analytic process 
1s similar, but more complex, since 1t may be more difficult 
to determine problem areas. 

A comment on "other approaches" 1s required here. If one 
method poses a problem, another method may not have that 
problem. For example, assume that for a particular course, 
analysis has revealed that the learning of Ohm's Law 
presents a problem to many students. An examination of the 
objective might reveal that knowledge of Ohm's Law 1s not 
really needed 1n order for the Individual to perform his 
job. If 1t 1s needed, the parts of the objective are 
examined and then the way 1n which 1t is taught. Assume 
that each part of the objective is valid and the method of 
teaching appears to be reasonable. Now other approaches 
come in. A nomograph might be devised for calculating 
problems, a simple mnemonic might be applied, or a conver- 
sion card. Here an acceptable performance aid might salvage 
trainees that may otherwise fail. Such aids should be re- 
ported to CHNAVTRASUPP for cataloqlng (see Function 8.0). 
Also, as appropriate, aids of this type should be made 
available to technicians in the Fleet. (See Function 5.0). 

The place of training/performance aids in the idealized 
training situation was mentioned previously and 1s covered 
again in the context of the discussions of Functions 5.0 and 
8.0. 
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Such aids can be valuable m the training process, 
but equally or more so, 1n actual Job performance. 
Training of Instructional personnel should Include an 
Introduction to aids, Including the principles of 
. development. 

Aids which are developed and verified through trials 
must be made available to the total naval establishment. 
The appropriate agent for this 1s the CHNAVTRASUPP (see 
Function 8.0). Also, when students finish a course, each 
should be given performance aids and reference materials. 
Such materials would be an Integral part of OBT as well. 

16) In the light of great pressures to Increase cost-effec- 
tiveness, 1nter-serv1ce schools offer an excellent poten- 
tial solution. Cost studies should be made to determine 
1f other service schools offer cost advantages, assuming 
the content meets the need. Civilian training should be 
examined also. 

Similar effectiveness needs may be met by Improving facili- 
ties usage. This might be done by Installing more load- 
complementary courses within a single facility, and/or 
scheduling facilities for other activities, permitting an 
overall reduction 1n facilities needs. 

The Implementation of the earlier approaches will also 
have a decided Impact upon cost-effectiveness. 
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5. Function 5.0 Evaluate Performance (Feedback) 

a. Definition 

This function deals with the determination of how well 
training prepares the Individual to perform 1n his job 
assignment, Thus, 1t 1s post f ormal training feedback . 
Information about how well a student 1s learning, or 
Intra- traini ng feedback 1s placed 1n Function 4.0 
Implement training, since such Information 1s an Integral 
part of the teaching process. Certainly knowledge of how 
a student 1s progressing is Important. But this does not 
tell 1f the skills that are acquired will enhance his later 
job performance. 

Feedback 1s a vitally Important function, since the absence 
of such knowledge could result 1n failure to accomplish the 
training mission -- preparing individuals to perform on their 
job effectively. In spite of such significance, 1t stands 
as the function which 1s 1n the greatest need for Improve- 
ment. 

The subfunctlons and their Interrelationships are shown in 
Figure VI 5.0. After the determination of which method or 
methods will be used, the method 1s applied. Data is 
gathered, analyzed, and evaluated; and a decision is made to 
revise the curriculum or course. 

A major problem became apparent when describing the "current 
system." In the process of analyzing the field data it was 
clear that some segments had not been mentioned by the 
respondents. In many cases, 1t was impossible to get from 
one status or condition to another without a number of Inter- 
mediate actions/operations. Consequently, 1t became neces- 
sary to develop such intervening operations. In some 
Instances, what 1s being called current may not be performed 
In reality. In other cases, only one example of a subfunction 
or task was fownd. It is Inappropriate, where this occurs, to 
state that such a task 1s characteristic of the current 
system. Therefore the flows for Function 5.0 contain a 
combination of both "Current" and "Idealized" subfunctlons/ 
tasks. The Idealized ones are so marked. As mentioned pre- 
viously, some of the "flows considered to be current, may 
contain elements which are not really being performed ex- 
plicity, but certainly should be. Those blocks with digits 
preceded by an "N" and followed by an "A" are new (idealized). 

If the block has the same number as a current block, but 
is used in a different sense, the digits are followed by 
an "A". Those flows which contain all idealized blocks 
will be so marked at the top and bottom of each sheet. 
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The flows thus represent a combination of curreht and 
Idealized approaches which follow the present training 
system context. However, a more drastic change 1s needed 
In order to obtain the best possible feedback. This 
approach will be outlined 1n a later portion of this 
section. 

b. Description 

The Evaluate Performance (Feedback) function 1s described 
through an outline which contains all of the functional 
elements presented 1n the functional flows, and 1n addition 
through an overall functional flow, Figure VI 5.0 - Evaluate 
Performance (Feedback). 
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c. Functional Description of NETS - Function 5.0 Evaluate 
Performance (feedback) 

A general description of this function was presented 1n the 
outline and top-level flow of the preceding section. Further 
detail 1s shown 1n this section through a series of flows 
which expand key subfunctlons. In addition to notes within 
the flows that elaborate on significant points, discussion of 
the flow 1s contained on the facing page. 



9 

ERIC 



VI -1 85 



TAEG REPORT NO. 12-1 









to 




:ion mu 




formal 




c 




•r- 








u 




! 








c 




•r- 




C 




















•8 








J> 








to 




-o 




O 




eth 




e 
















to 




to 






u 


a 






N 




•f- 


o 


r— 




*J 




<D 










> 














a> 


U 


to 


•r- 








S 






o 


to 


-M 






to 


to 




TJ 


ro 


O 




-C 


4-> 


i i 




01 


SZ 




+-> 


a> 




c 


i 






i 


x: 




CO 


<D 




+-> 




0i 




O 






c£ 




to e 






• 


£2 



o 

ERIC 



VI-186 



TAEG PF.PORT NO. 12-1 

» 



$1 
£2 



J5 

UJ UJ 

OK 



» I 
* I 
I 

U. 1 

Ui I 



ui 



ft 



o 

uJ UJ 
X * 

lA O 

< x d 
in • t~ *• 



ft? w 

o o 
u. * 

o o 

Ui Ui 

Ui ui 
X u. 



"1 t 

"I 

U» I 

UJ 



ft, u 

ft. UJ 



I 

I 

I 



ft I 

f* I UJ 

I I -J 

p4 I UJ 

• I iA 

X • — 



O 9 

tt. M 

UJ U 
ft, Ui 



• — 

< 
I 
I 
1 

I 

I 
I 



r: 



UJ 



ft I M 



4 

N 



p4 I UJ 

• I o 

Si- 



# *- 

I 



231 

IA UJ 
UJ * 
O «i 

ss* 

o O X 
Ui i/l o 
UJ Ui < 

2 « ui 



•J— — « 

i 



< i 

• I uv 

t a 

< i m 

» I * 

~4 I O 



I 
I 
I 
\ 

t/J I 
J Ui I 
< X I 
»•« U I 

53 ! 
Hi 1 
25 



7F" 



I 

i 

a? 

M 

X 

at 

UJ 

»- 

s 
i 



Ui 

3 



Ui 



Ui 
UJ 



X # — 



J 



ft 

ERIC 



TA£G REPORT NO. 12-1 



a 

N 



OJ 

x> 



CO 
OJ 

JC 

u 

O 
i- 

Cl 
Q- 
< 



■4-> 
C 
<U 
■4-> 
O 



r- 0) 








4-> -P 




c to 




Pi 3 




o 








r— 




JC X) 




O rt$ 




«J r- 




<y •r- 




u 2 




If. > 




O 4tf 




to to 




OJ OJ 




on u 












c o 




<o to 
















to OJ 




•r- x: 










t 


c • 


c 


A3 C 


o 


O 


•r- 


00 *r- 


4-> 


OJ 4-> 


U 


CO o 


OJ 


<d c i— • 




0> 


C <f- 


to 






> to 






*<5 


* JC 


G 




• r" 


OJ 


+-> 


5^ 


do 






U 


c 


OJ c 




X3 O 






x> 


</> 


1- 


C O 




O OJ 




o to 


o 










4J • 




OJ 




to^ 


<xS 




+-> 


^ c 


to 


c «r- 




o 


o 


»r- TO 





4-> OJ 
UP I. 
C (G OJ 

VOL 
jQ JD O 

to c 

OJ *r- 

00 







to -o 




JC 


•r- OJ 




+-> 


V- 


-o 




tO <D 






-o 


to 


O JC «r» 


c: 




to 


O 


0> 


c 


O 


o 


o 




i- 


• o 






< 


to 


O t- 




CO 


03 0J 




o 


O JC 


i — ■ 




U «M 


■ 


OJ 


Ol S. 


to 




Q. 3 




h- 





9 

ERIC 



VI-188 



TAE6 REPORT NO, 12-1 



i si 



I 



1 2£t; 1 
S3 

UJ V> Ui 
X UJ H 

ISSSi ! 

i UJ UJ UJ < I 

1 ft X U uj I 



W 

4 — 




* « X 

• U M 4 

tt ui r- 

ft ft \A 

• Ui 4J *"> • 

uj ui x 
• u. r 4 



t 

4 — -L — — * 



Uj fc* UJ 

Z UJ J 

mo as 

* * -* 

of 3 J 
UJOm 

H UI 4 

I Ui Ui > 

\ Q0C < 



* • — 



2F 



-J 

Ui 

ft 



oo 

< ^ x 

D * 

-J ft uj 

< at Ui Ui 
> O Ui * 

ui u. u. 



* — — — # 



I \A 

1 < * 

l ui 4 

1 ui z 

I K o o 

1 « * Ui 

N I 3 O Ui 

« i a 

» 4 ft. 

• i > a. o 

^ I UJ 4 H 

• f 

^ # — — — — 4 



a 

Ui 

i 



I 



& 
ft. 

* 

I 

Ui 
Ui 

u. 

* 



• 4 • • ft 




4 

1 




ft 






» 
1 


H X 1 








1 


x o i 








1 


^ < i 








< 


> O 1 










ft af l 








< 1 


< ft. 1 








w J 


i/> ft 1^ 




+ -J UJ 




ft 1 


~ 4 t< 




— (MI 




1 


Ui O 1 




* M O 




* 1 


»- v» I 1 




*-* ft * 




~4 1 


< »- O I 




« I O 


ft 


• 1 


S X «< 1 




M-l • V) UJ <X 




M 1 


-J UJ t 


i 


X ft X ft 


• 


« 1 


< > 1 




# ft ft ft 


ft 


■*4 I 


> ft u, t 


i 


u. 4 u a < 


ft 


• I 


Ui 4 O 1 


i 


Hi • 


4 


* 1 




i 


*«•••• 


ft 


X ft 


— — — 4 


• 



I 



> 

Ui 

5 



Ui 
Ui 
SA 



© u 

U. 4 

tt 

ft ft 

ui Ui 
ui Ui 



u. 



> 

Ui 

I 

ft 

ft 

Ui 

ft 

UJ 
Ui 
X 

\A 
V 

5 

-J 

wo 

1-4 
* 

ft 



o 

X 



s 



u> 
o 



i 

ft. 



X 

Ui 



2 
2 

O 

I 



UN 
X 



Ui 



ft 

ERIC 




"8 

0> 



C 

0) 

E 

•r— 

i. 



CO < 



3 

c 



r— 
r— 

2 



co c 

3 C -C 

2 0) 4? 

o c 

4-> O C 
C -r- 

a c 
« a o 

r-» 0) r- 
C 

c o 

0> <V <r- 
E C <P 
•i~ »r- O 
i. E 3 

a? E j- 

X CO 

3 a) c 

-r- 



0) 



C 

c 
o 

*r- 
0) 



3 
O 



C 
♦r- O 

2 
§ 

CO 

? 
c 

§ 



CO 



<D 00 4-> 

^ *> <P 

gl. CO 

CO 3 
■p* CO CO 

O 'i-- c 

C OJ o 

C <o 
O i- 
<+- o c 

0) -a jc 
a c -p 

O CO 

o+> 

l» Dp* 
4-> 3 3 
r- CO ID 



<0 

o 

s 

c 

<0 



f 

0> 
CL 



o 



oca 

0 o »r- 

£ co a. 
a. 

<tj ao 

"° S CO 

-o a. . 
f- <p ao c 

2 * g\= 

0) CO 'r- <0 

<J <TJ 4-> 
CO C L> 
0. 0> 'r- r- 
D'ftJ O 

01 J- C Q 



e x 

*j o> 

•r- o 

♦r- O 

"6 co 5 

0) ***• 

h— SZ CO 



CO CO 

c 



CO 

§ 



CO 

8 

0) 
OS 

2l 



CO 

s 

o 

00 
•r- 
00 

c 

< 



s 



0) 

■o 

O 

c 



8 

c 

•r- 
+J 
CO 
'r- 

X 



4-> CO 

D v» a. 

lu t in 8 

o 

0) U. i. c 

JC 'r- x: 

, CO Q. 01 
<*- 0) 0) h- 

o 3 a: 

0) O CO 

<^ at s 
>• o 

3 a.* 

p* sa 

> C 0) 

»r- »r- 0) 0) 

c cp-o w 

•r- »r- C CP 
flj <TJ »p» 0) 'p» 



8 
o 



i 



VI-190 



a. 
I 



3! 

I! 



St 




3 

rO 

> 



8 j 

•r- «r- 
X> 

Out- -r- 

V> C |r- 

Q fO <t) 
Xi 25 oo > 

r— rO 'r- rO 
t)3BC 

O rO o 3 

a. o >- o* 



2 c 




TAEQ REPORT NO, 12-1 



4J \AOC 



3 3 

>>r- 
r* *r™ 

rO > 

n <o 

oo 

♦J £ 
C T3 
rO rO 

O 

c c 

►*-« ro 



T* I— • «0 "O r- 

+-> a -p cTj xi 

I U 00 



x: 

«+- 
o 

00 

.2 



8S? 



ro 
o 

0) 



ro 

s 

c 










c 




c 




o 




00 






a; 
c 




c 




o 




uO 
















On 




C 




c 








/a 


C 00 


t: 


o c 


•r- t~ 




A3 O 


a' 






O T3 
c C 






o 


C7> 00 


c 


c oo 


o 


• r* ft! 




c 


+-> 


•~ >1 


0 


<0 r— 






O 




QC 


o c 



iX> 



U 
rO 

-a 



rO 

E 

o 



T3 

2 ' 

U C 

•r* 

C ro 

C OO 

o 4/ 



V r- 
r— 

«*- I. 

o 
§ XT 

3 
ro | 

ro 

o o 



s 

3 



00 



00 



c 



00 



e 



> U 



i 



r» C O 

C <ti +-» c 

5 J- ja o 

M rO +J 

t> rd 4) T> 
•O C X> 
rOOO.ro 

yo v> v> 
O i- 

O'f* 'r* 
3 ( « 3 3 

8" U % % 



c 


4-> 






•r- 


<+- 




o 


-M 




U 




r0 


J 






§ 






c c 




C -r- 


ect 


O 




a c 




ao 







0) 



r0 



O 



O 



00 
> 

o 

V. 



•r- OL 
r0 to 

<u o 

Q..C U 3 ro 
O (A oo g T3 



x: ro 

O 4~> 



ro 

ro 
T3 

rO T3 

(O r« 
ro 
> 

to ^: 
oo oo 

ro 

r- £ 

u 

00 

o a 

00 O 
rO S- 
LU CL. 



^8 

3 
O 

2 

CO 



00 

00 
• r— 

* 

00 



c 

c « 

O 'i- 

c oo c 
cox: 

rO 3 U 
r- CT <D 

a. +-> 
■o 

rO O O 
C 3 *J 

00 4-> -Q 
•r- 00 O 

>^ 

5 



C 

c 
o 

i2 



* c 

00 O 



<1> 00 "O 
Q. ro <U 
00 

T3 «0 

C i 



rO 



a. g 
% o 



CM 

oo 



r— ■ 

a: 

CO 

&. 
o 
a. 

C£ 

sz 
o 
&. 

rO 

<D 
00 

2 



a. 

2: 

CO 



•1— 
c 

♦r- 
rO 
i. 



rO 



ru 
O 

00 
00 

8 

u 

rO 

<D 

r 

XI 
rO 
U 



Q. 

a. 

rv 



o 
c 

>v 
u 

rO 
X> 

o 
+-) 

00 

<u 
x: 
u 

rO 

e 

Q. 
Q. 

rO 



ro 
C 

o 



T3 



a. 



+-> 
rO 
3 
r— 
rO 
> 



XI 

c 
<0 



c 
c 
o 

00 
L. 
<D 
&. 

S 

CO 
00 

c 
o 
a. 

re 

o> 

3C 



c 
c 
o 

£ 

CL 



2 

o 

c 

o 



rO 
Q. 



r- S *5 



4-> 

JC rO 



r- CM 



UJ 

•1— 

— > CO 












t/Ty 


•r- 
C 


• 


<D 


•r- 


00 


<D 






r— 


2 


'/> 






O 




r— 


U 


rx 


O 


0) 


3 


O 


ex 


ro 


■5 




CD 


CO 




<D 




<+- 


CJ 




O 




O 






00 


+J 




■r- 


00 


c 


> 





ro 
•r- 
U 

00 

c 
o 

T3 



x> 



CM 

3 r- 

f & 



3 

•r- «r- 0% 



CVJ 
1^ 



CM 



^ 00 2 

00 2 

C O 
•r- LU 

C -r- CO 
•r- r- >► 

rO rO «£ 
U 3 2 

h~ CT — ^ 



ro 



VI - 191 



9 

ERIC 



TAEG REPORT NO. 12-1 



o 

T > 
C £ 

o 

i£ a 

P (1) 
U 

c 

o c 

•r *f" 
P 

SCO 
•r* 

§8 

+-> 

U N 
3 »r- 
00 C 

§° 

•r- CD 
uj +J C 



N 



< 



u 



</) 
0) 
CD 
<TJ 
■P 

c 

00 
•r- 
O 

0) 
CD 
*J 
P 

n 
> 

0) 
P 
fO 
3 

> 



<0 <T5 

> U / 
<1> P U 
<D 

O -C 
P V) 

-O <*- O 
<U «r~ O 

C * O 
<D P 
0) r- 

x: a. i- 
-p H o 
<a p 
VI xu 
dJ d) 

N <«- 
WOP 

<ou c 

CL U 

g« «r- 
P <+- 
X »r- 
<l) C 
<D P CD 

r 

P O to 

t- u 

X 55-° 
o o 



IT) 



o 

•r- <U >> 

x: a. S? 
h- o E 



ERIC 



VI-192 



TAEG REPORT NO. 12-1 



4> 

4 ♦ 4 

• * * 

« < * T 

> > O 

-f rs 

« << UJ 



» H i*i O 

< ft 5WsAO» 

V ft ft 
ft 

* # *J »- w -« * 

• ft 



I 
I 
1 



4> 

I UJ O Ui 

r uj j 

• w U • 

m r <y « 

I I Ml D *J 

y i hv^< 

I UJ W 

i Ok < 

ui I 

* * 7G" 



4 I 

I 

« I 

^ I 

• I 
M t 

• I 

^ • 



UlO w 

> &. r 

« O S 

M UJ «tf 

W 4 



7F* 



* — — 1_ w « 



» I I U. 
I < 4 O 



I 

< I 

N f 



> > Ui 

CO a 
* 

M V> l/> UJ 

* O O * 
0446 
U m ►* O 



>~ *4 i 

ft * » 

MHI^ | 

t* »~ UJ * 

vuuj r t i 

>- ' U o I 

« o t- 4 t 

o 4 n o i 

j v & ^ i 

-4 * <X I 

> * 4 4. ( 

Uiu0 4 I 
t 



Ui 



u 

8 



Y 

U 



UJ 

I* 

T 
> 
< 

M 

O 

<A 
Ui 
<9 
4 



4 

4t 



UJ 

I 



O 



o u 
u < 

t * 1 

I QO » 

l lual 1 

I Ui Ui I 

I 3? U. I 

I I 

» ft 



9 

ERIC 



TAEG REPORT NO. 12-1 





oo 












mo 






res 






i which 






c 






^* 






At 


















^* 


















V 












V- 






r ™ 


















•V 






•"1 






























XJ 






TO 






r* ■ 










<D 


u 




N 


JC 




'r~ 


00 




r— ' 






fO 


XT 




<y 


O 






*r* 






JC 




*" — > 


3c 




<D 


00 












o 












U 




•r- 


























o 






•r-> 




0) 






O 






1- 












SO 






<D 






or 


00 










OJ 






c 


x: 






oo 




£ 


c 






o 




4-> 


•t— 


• 


<D 


4-> 


<u 


O 


U r- 




C 


XI 




t3 


rt* 




4~ 


r— 


<t 


-O 


*r- 








r— 


00 


> 


• 








00 








(U 




x: 


t- 









9 

ERIC 



VI-194 



TAEG REPORT NO, 12-1 



ft 



ft 



o ft a 

kA M fi A. 



I? < u. < 

„t_. 



4 < 



g 



I ft uj O I 
i u n *• at i 



i > 
M I 
I 
I 
I 



I Ul ft <» 

* r * w 

m n m ft 

wo * 5 uj 

UJ « V- * 



— • « 



> 



71 



8« 

M ill 

T Q 

m UJ 

4 u 
erf r 



<£> 

M 
> 



UJ 

>- 



UJ 



UJ 

* ft • 

X UJ 
ft ft 
Ul UJ 

* UJ • 

uj r 
• a. » 



Of 

o 




ft 



UJ 

ft 



UJ 



3; 



i 



O 
UJ 

a 

UJ 
UJ 



I 



Zu)Xl 

* I * X M 4 

i \ u o uj a 
h i w / * 
I I t- * <* ft. 

M I UJ UJ OA, 
« I O dL U, 4 

Si — * — 1 



• # # # 



« UJ V) UJ 
/UJ J 

• mUD 
~* * * < 

• • of O -J 

• Ui ft M 

y i h <a 4 

• UJ UJ > 
U. • ft * * 

111 • 



. — -J— — i 



! 


UJ 1 




mi 1 


-1 


a i 




IB* «S 


• 






J VllU 


• 


UJ O — 




> 6. T 


• 


O 




>- < 




< a o i 


$ i 


►* UJ « 1 




2 -J <X 1 


1 


UJ UJ 6. 1 


U. 1 


1- 4 1 


UJ 1 
ct • 





ERIC 



TAEQ REPORT NO. 12-1 



P 

<D 

o 





•o 












I cula 




be cal 




< • 




3 




& 
on 




CD 




P 




Cl 

o. 


& 




N 




»r- 


a> 


r- 






-O 


CD 


*r 


■o 


on 




on 




O 




Q- 






> 


P 


•r- 


C 


P 


o 








o 


$~ 




<u 


*+- 


P 


•r- 


r— 


"O 


< 








x: 


,p 


u 


4- • 


<tj 




LU 


T> CD 




<D U 


i- 


"O 1~ 


o 


CD 3 




CD O 




C 00 


on 




<D 


00 $- 


U 


CD 




O C 




*- .° 


o 




00 


O 


(11 


ui on 


a: 






C a) 




<u 


<u 


CD C 


"D 




CD 


•P CD 


<u 


x: 




P P 






<u 




c 


P o 




on P 




5 O 




O <T3 


+-> 


CL 


a> 


on e 


Q 


•r 








o <u 




r— x: 




+-> 


» 

r— 


on .c 




•r- p 






2= 


h- 5 



9 

ERIC 



VI-196 



TAFG REPORT NO. 12-1 



uj 



u 
it 

a 



< ~ 

<S X t£ X H 

• ** o < 
* a « * 

• ft ft Uj uJ * 

t uj a uj 

* ft *-< ui <* j 

ft iUUO Jf 

a • o * u. < ft 
ui ft ft 

* ft ft • ft ft 



t H ft 



4 

1*1 



•« a 



£ <J X 

<P I mJ ai u 

I UJ & UJ 

U» \ s/l v> t» 

ai- 



1 • 

i 



O 



• f 

• I 
~* I 

■ I 

Z # 



I- u 

~j < o 
« e\ 
> r uj 

7\" 



I 



t 
I 

i . _ 
I V) O O 
I vjj K 
I CO 2 
1 O VI 



2 H X 
-* < SJ 

X z < 

tt o 

UJ 05 of 

M 5 * 

uv O <v 



cx 

UJ 

>• 
a 

IN 
* 

I 

I* 



UJ 

o 



. » , 



V> UJ X I 

o > u » 

o < UJ I 

uj > I 

U. U IM ^ 

a: uj * h 

« U. O < I 

t UL U. Z I 

ac uj tt \ 

UJ O UJ 4 

H O sA >- 1 



I . 

I UJ X UJ -J . 

i o « i « i 

• — — • 



• — — — —ft 



Of 

O 

. v Ik «J 

4 I w> < 

* O V 
» UJ ► #• 

* IU « 3f 
- • Z JJi 

— >- x 

« UJ CO O o 
M 0C *m Ok % 

* I * ^ o t 

u> i o < uj < j 
uj i i 

U ft — — — — ft 



9 

ERLC 



TAEG REPORT NO, 12-1 





» 




■o 

e 






• 


6 




-o Jo 




O to 




r; ^ 








<jj <j 




f? at 

e 








O (/> 








5 x> 




4J 4_> 












+3 
















o 












O 10 












o <u 








1— • d> 




0> "O 
























o> 




Q) 




r-» $w 






<D 






4-* (/I 




C Ct 




*<— 






(V 


C t/) 




CD 




>s 








lA o 






V/J 


i A #rt 
vi 'V 






rt * 


r— ■ • ro 


■mJ 


*o O. 


rr 


« 2. 




1 V 


V- 


t\ 








f* c 

V— » ■ 


At 


O 


I— 


VJ »> 




(J J M 




M S 

<-> fcr 




fli 














rt * 








oo 






C 


+-> 


4- o 


U 


O T> 


O 




r— 


4-J lA 


<U 


x: 


00 


CD 




•r- (/> 




r — ■ »r- 


< 




CO 


<D 4-» 




x: 


r* ■ 


+j a) 




o 




c c 




o 



9 

ERIC 



VI-198 



TAE6 REPORT NO. 12-1 




• • • « 



4 Ui 
t MO 

f • U " £ # 

sei juio 1 

• Ui O. Ui • 

u. # * • 

UJ « • 



* # 









» 














Q 




< 


UJ 






Ui 




X 


z 










• 


u 


< 








IU UJ 




o 


K U 




UJ 






UJ 


53 




U> 




o 








AC 


O UJ 




o 


Ui oC 




a. 









4 

1*1 



u, 

ui O uj 

D U 5 ^ 

< tk Vi U) 

w -* a * 



<* « — — — 4 



o 

ERIC 



TAEG REPORT NO. 12-1 



3 

I 

g 

IS 

CO 

3 
I 

C 





4-> <D 




10 CO 




% 3 




C 








x: 








o 




<D SZ 




i- «M 


<v 




CO 






* e 




CO c 


CP 


0) o 


ja 


jc g 




■p E 


o 




+j 








to 




X) 


0) CO 


o 


o o 


sz 




±> 




0) 




>: 








<x> 




c 


c 




<D , 


e 




c- 


S*. 0) 


<u 


13 C 


4-» 








Q 






xr jo 




+-> o 


i — ■ 
• 


C CO 




H- 1 «r- 



9 

ERIC 



VI-200 



TAEG REPORT NO. 12-1 




ERIC 



TAEG REPORT NO* 12-1 



0) 



8» 

Q-r- 



O 



c 

o 

to 

t 





* 








e • 








to 




e 




o 












O 




• <D 




Yl 10 • 




<D <D 




00 * 00 




•r- C 3 




> O , 




ai r- v. 
















g 3 «M 








C 




OJ 00 O 




C «r- <f- 




•r- x: 




rd 4~> oo 




c *t> 
CUO 




o ox: 




• r- If, +J 








i/i c E 




<D O 




3 V- T> 




CT 00 0) 












o o 




x: oo 








•r- "O 






00 


<u o 


0) 


x: 




C h- *o 


Or 




•r* 




C 


x> . o 


x: 




u 


> <D 0) 


4> 


£ s~ > 




<D U <U 


<u 


00 3 TO 


xi a 


to 


o U 00 


•1— 


O r— 


> 


00 O 


0) 


<t> (/> o 




5 c x: 




o o 




<D -r- 00 


o 




r— 


C KS r- 


0) 


3 


> 


«P 3 




t/> "O 


o 


C 00 «r* 




•r- > 






CM 




• 


C 3 C 


to 


O 00 «r- 



ERIC 



VI-202 



TAW REPORT NO, 12-1 



♦ 
# 
ft 
ft 



» - 

I 
\ 

I 

I 

i 



mmmmwm ft 

I 













i 






1 ^ 


a 

M 


\ 

1 4. 11 


1 ft 


UJ 


t 4 w 

i 


•—7 


r 



ft 

I 

J 



T ** * * 

v- r x o 
r »- 
, v o w 

» UJ u UJ UJ 

iz « 

a i utuar . 

• i w lu uj a i 
<v i « x <a o i 

• « i 

lA • — — ft 

A 



U. • 

O ! 



3 

Of 



I Ojuj 

I .J m t* 
I UJ 4 
I > K UJ 

i uj uj r 

<u ; 





SA 








i 
l 
















UJ 




< 


UJ 












a n 






a cr 




• 




< 




Oiuw 






W U. M 


l 


M ft 


•J *A X 


ft 


• X 




I 


• ft 


UJ M X 


♦ 




ajor i 




WS ft 


> > O 


• 


81 


JUL) | 




ft 


UJ «J UJ 


ft 


1 


uj a. uj 1 


41 


U. ft 




ft 


U. 1 


t* n i 




UJ • 




• 


UJ 1 




• 1 


ft ft 


» * ft ft 


ft 


<* • 




k/N ft 



3 
ft. t- 

O ft 

•4 w 
UJ u 

> X 

UJ o 
o o 



— * X O 



Q 

I * 

f UJ 

15 



uj w 



u o 

O Ui 
N 

M 

Si 
O uj 
Q 



UJ 

x 
< »- 

X 

u. 
a 

*- 

a X 
« < 

u. x 

UJ 

X at 

UJ 

X 



Ui X 

r> * 
o >* 

UJ t* 
X 

o uj 

S3 

a 

UJ 



as 

3 < 

u. >- 

* O 
fr- 
iz* 

■M >m 

X ~ 

UJ UJ 

£ « 

* u 

i 

w 
>- 

O 



3 

Im* 

I 

O 
UJ 



UJ 
sA 

M 
> 

UJ 

at 
s 
ft 
O 
w 

UJ 

> 

I 

N 
« 



Ul 



9 

ERIC 



TAEG REPORT NO. 12-1 



i 



JQ 

"S 

c 

•r- 
C 

♦r- 



XI 

o 

X) 



crv 
c 



00 

D 

cr 
■r- 
c 
x: 
u 



' r- 
> 
I 

$- 

> 
o 



Q. 

n 



ERIC 



VI-204 



TAEG REPORT NO. 12-1 



Ui Of 

< * 



* ft — « 

A 



U. j 4 0 

uj I 




I 
♦ 

I UJ 



I 
I 
\ 
I 

• ! 

I 

** I 

• I 
"> i 



* UJ 

53 

•4 1-4 
> * 

A. WO 

«. a uj 

4 Ok- 



i 

f 



I 
I 
I 

-J I 
I 
I 
I 

\ 



X 

o 



\ 



Of 

< UJ A. » 

* r a? i 

*j * _ i 

< O X U. I 

w. u. o < I 

I 

Omu. VI I 



ft — -J— _ ft 



ft — -J— — • 



o <v i 

I- U, I 

- S +4 I 

M Of ►» I 



I 4 O t h I 

» I « * of ~ I 

* I o ** u. > I 

4 1 I 

* ft — -v- — • 



•J 

si 

u. 

« * </k 
ar t mui 
m at U 
< o z * 

O «-« U. 



"7T 



• o or 

y Q 



-A 



! i 



*3 
O 

«# ►? 

>• Jt 
Vft • A. U 
# Ov UJ 
u. # < »- 
UJ # 





or r 




Ul o 




Q + 




« o 




*~ ct 




9. V> O. 




O « 6* 




O 6* * 






or i 





UJ 
Of 

or 
• Ui 

g* 

►* uj 
UJ u 
IT * 

,£ 

O 

K * 

s: 

U X 
o 
tu => 
X * 



Of 0) 
UJ UJ 

</> * 

UJ 3 

t- UJ 

Of 

• < 

*§ 

uJ 

O 

\A 
U UJ 
< \A 

to < 

o u 

UJ 

UJ O 
U. UJ 

UJ 4 
Of -J 

I 

UJ 



1 



o 

UJ 



Ok 

o> 

1 



UJ 
Of 
D 

u 



o 

ERIC 



TAEG REPORT NO* 12-1 



> 



03 



«r- 
C 

'6 

JC 
KA 

<D 
U 

1 

o- 

<D 
t/> 

<D 



• O 

C 4-> 
CX 3 

x cr 

01 





0) 




<f- JC 




r— 4-> 




01 




KA 0) 












C 3 


cr 


03 JC 


•r- 


O 


C 


</> *r-> 


JC 


0) JC 


o 


+-> 5 




O 




C L0 




c 


<D 


C7> O 


$- 


C »r» 


•r- 


4-> 


03 


>> 03 


C 


C N 


c 


03 


o 




•r- 


e m 


4-> 


O cn 


ui 


a 


<U 


o o 




03 


O 


cn 




x> c 








03 C 


a 


♦r- 


CL 


</) 03 


< 






|h 






r — • 




• 






0) o 




x: -c 


in 





9 

ERIC 



VI-206 



TAE6 REPORT NO. 12-1 



# mm mm m 

1 



Ui t» Q 
uj ft. X W 
oc a u. w 



N 
>■ 



• — — — — « 



UJ 1 



*2 

32 



1« ui 

i u> oc x r 
* i w o *- \A i 



»Q of 
O 



•a' 



i 

S 



fit 



• — — — — * 



I (3 

I Ui X 

I Nf Ml 

I «J Ui X 

» tr k o 



i Duiu. i 



o 



ft, 

O UI 

-I I/) 

UJmX 

> > oc 
Ui ui O 
o oc u. 



^ # — — ♦ 



£$ 5 

< o. uw 

% n 35 

fltO z 

a u v* 

u» fit M UJ 

x uj 

Ui o 

ui 3 oc ~ 
O K O OC 

u v> 

Z Z « 

UJ M UJ > 



tf\ • — — — — • 



UJ 

oc 

on 
< 

X ui 

- S3 

• H 14 

r* • > *■* X 

• • mi VI X 

* ft a. ui o 

• * D Ui 
U * < f | #_ 
Ui • 



( 

UL I 

ui I 
u # 



Of X 

UJ (J 

D < 

M O 

Z \A CL 

a < a 

o ft. < 











Z 








SS 


o 










o • 


X 


UJ 




— ft. 


Z O uj H 


f>- 


Z 




H 


O UJ X 00 


z 


o 




OC 


O »- 


o 






-J Ui 


u> «t 


X 


• 




-i 


OUfiCVI 










UJ z o o < 




o 






h « J U, J 




-J 


VI 


H Ui 


< z 








V> 


OC Ul UJ Z 








31 < 


< Ui U^ oc 






< 


UI 


r or >- < d 


UJ « 


UJ 




M OC 


OC K < 03 


1 2 


> 


u 




D 3 ui 


< O 




Ui 




OC O mt OC 


or « 


v> ft. 




Ui o < < 








u> 




UJ 






a z 


U O < o 


UJ 


Z 


■J 




D Z 


X ft. 




< 


z* 


ui OC O ui 




§ 


z 


oc o 


UJ K 2 Ui h 


H O 




o 


3 Ui 


X ^ M I < 


o 


\ii 




H >- 




Ui 


oc 




UJ Z 


MM (J 








OC UJ 


X * ^ 2 " 




u> 


u 


UI 


Z OC < n 


o 


O 




< Ui 


uj < r> uj x 


> X 




o 






-J o 


Ui 


Ui 


o2 


H X 


-J o 






ui uj 








•J 


H fit 


• or -i oc mt 


or fit 


OC 


o 


OC 


ui u. a uj o 


UJ Ui 




o 


O ui 


oc o > o 


z 


UJ 


X 


ft M 


<wi r x 


UJ u. 


X 


o 


uj X 


Ui < U OC O 


o < 






OC 




1 

mi 


i 

<v 








UJ 


UJ 


















o 










at 


z 









ui 

i 



! 



UJ 

3 



ERIC 



TAEG REPORT NO. 12-1 





« 

sz 




0 

e 




a. 




0. 


c 


«j 


c 




o 


to 


t/> 


•r- 


U 




01 


+-> 


a. 






l/> 




<U 


t 


N 


CO 


•r- 


+-> 


s- 


00 




3: 




a> 




2: 


00 


B 


<D 


O 


+-> 


$- 


O 


U- 


c 




en 


O 


c 


> r— 


•r 


+-> 






C 


s 


A3 


i- 


Cl 


0 


§ 


<+- 




c 


u 










c: 




• r~ 








+-> 








O 






O 




r— 






* 

OO 




» 


x: 







9 

ERIC 



VI-208 



TAEG REPORT NO, 12-1 



O Ui mi 



w <n mm «. 

uj ui uj m 
> 5 * -i 



Ui 

, uiew 



# ft • # • 

ft 
• 

-I • 
Ui* 

x » 

* s 

*+ « 
*- 9 Ui 

< Ui ft* 

♦ r t x 
<•% • * u> • 

« • 40IH.I 
^ • NlkO<« 

UJ • • 
* • # • * • • 



i — ft 



Ui 

15 

i » 



• I -J & 
N 1 * i/> »/> 

l I > WU 

Ck I UtK 

• I 
* ft — - 




i 
i 

j 
» 

if I 
v Ui I 

Ui A I 
If X I 
* UJ I 

o o x i 



• I ui s\ x wl i 

* I J 



< x o 



Ui 

Ui 
X 



. OUJ 

I Ui M 

i* i * *# ui u. 

I j UJ A. x u» 

« i wi ui < 

• j eo>t- 



SA 

* Ui 



* 4. D I 

^ O O UJ Ui I 

* JHlrtO I 

I | UJ tf <t X 1 

• U»90X i 

« i i 

*N ft — — — — ft 



— L--< 











at 


UJ 

r 




UJ 


u 




o 


4 


m % 


►* 


o » 


% 1 




« 1 


* 1 


§ 


l/> OW 1 


1 




< flw 1 


U, 1 


u 


A. * 1 


UJ 1 






tt • 




ft 









* 




UJ 


UJ 




X 






UJ 






UI 

-* 


& 








u. « 


< 


UJ 




X 


o 




1 


X 


* £ 




Hi 


Ui O 


UJ 




U t* 


X 




x 


>-» 


UJ 


« a 




a. 


u x 


UL 


X 




o 


Ui 




















O U* 


X 




X 


SA 




o 


* 


(3 


t Ui 


Ui • 


X 




G SA 


M 




< UJ 


M 




Ui W 


M 




•J 4 


•J 


u. 




M 


*> 


F 




* *j 


«t X 






£^ 








U, 


£ S 




O 








Ui *> 






3 « 


$2 


Ui 


O Ui 




u 






x 


Ui 


« X 


* 


H- ft 


iU 


fr- 




• sA 


iz* 




UJ Ui 




at 


> tf 




& » 






o v> 






S uj 


< 


UJ 






> 


&v 


M X 


or 


IS 


M 


w 




X 


i/> 




M ui 


0 




o 


O 


V M 






•J 


i5 


>• 


Ui * 






> Ui 


d </> 




«H u. 


X 


o 




> t> 




< + 


~ u 


UJ 




o oe 


X 


uj r 


X M 


1 


tt M 




UJ 






o 






* 







? 

o 

SA 



u. 
u. 

UJ 
X 

X 

o 



ftt 



o 



LU 



ERIC 



TAEG REPORT NO. 12-1 





T3 




e 
















8 




on 
















O 
















C 




3 
+-> 




O 




*~ 
4-» 




VO 




1 








o 




c 




>, 
















<D 
r— 

Ol 
E 




o 
u 




5 
s- 
<+- 






-M 


>, 


•r- 


C 








> 








>> 








G 


w 


<t 


O 

( 4- 


CO 


C 




« 


CO • 






E 


in c 


o 




S- 


c c 




'r *r-> 






c 


■a c 


o 




*r— 


C 




o -c 












c 3: 


O 




*f- 


£= cn 


C 


c 


k — « 


- 




5 r~ 


< — 




"r- 


•r- <D 


f0 


> XI 


+-> 




X> 


<D (0 


o 


<M C 




C O 




*r- «r* 


CO 


- 4-> 


• 




CO 


a» <u 








1- cr 



VI-210 

o 

ERIC 



TAEG REPORT NO, 12-1 



♦ « s ♦ 

« 4 



# — — 1- — . « 



, UJ X 



# — -J— — • 



%f> « — — — — 



52 




— — ft 



UJ w» v» 

«H 

> m > 

U^MUJ 

fr- u. o. 

2SS 



> 



o 



o 



2 

S 



s 



# * # • * 



i 

< 

x r 

o « 
o 
►~ II 

I* # *- * ft 

• • < O * -« ft 

* ft ►» U. O V* ft 
♦ CO JC <* # 

a* 3ni>« 

UJ # # 
Of « ft # ft ft ♦ 



■ 

m # 



> 

«* ar 

Ot uj H 

H X N 

•«* o *•* 

X z > 

5 o* o 

* « < u. 



ft — M ft 



Ui 

oc X 
Q ■< 

I +* o 
« I £ AC 



I 040. I 



> 



9 

ERIC 



TAEG REPORT NO. 12-1 





>> 




r— 








<M 




<TJ 




•r- 




V. 




Q. 




o 




S- 




Cl 




ct 












c 








>> 




r- 




+-> 




U 




t: 


iA 


*r~ 




U 


u 


u 






3 




o 






jC 




a 
















Q- 


O 


CX 




rO 


c 






00 




'r— 


e 


x: 


o 




t- 




U- 


O) 






c 


•r— 


o 


V- 




CG 


-M 






3 




CO 


O 






a> 


C 


+-> 




o 




c 


c 




•r* 


-o 


fU 


c 


+-> 




X> 




O 






o 




r— * 


•vt 








« 

to 





9 

ERIC 



VI-212 



TAEG REPORT NO. 12-1 



* 4 
I 

U. I 

iU I 

•c • 



Ui 

S- I 
is 

"TIC 



« — — — — < 



15 

x 



i ar 

I 3 



» -J Id 
I V» »- 

I Ui U.' 



Ui VI < 

v»<3 



I* 
I 

U% ft — -* * 



—J.— 



3 



iff 



1 i 



• o of 

• * o 
» 4 V 



oxo 

I ar if 

I *« X 

I UI 4 4 

i > x x a 

i ►«* ff at * 

I Ui U. 15 

I UJ X X UI 





• 




• 




« 




• 




• 
• 


o ui 






• 


* o 


• 




• 


u. ae 


• 














£° 








O 








M 








1- -J 








< <« 




• 










• 


- * * 


• 


• 


2 


400 


• 






*- u, U- 


* 




* 


X X 


• 


u 


• 


O ♦ « M 


• 


UI 


♦ 




• 




* 


• 9 • • 


• 



• — — L- — ft 

I I 



O 

UI UI Id 

> j x r 

I UiNt J 1 

I « v> < o I 

i ui © ar if I 

I a* a* ft> i 

< I 



ERIC 



a 

4. «i 

*s 

Ui 

i* 

ui o 

* 

uj ui 
0 u> 

UI 

5* 
U E 

>■ M 

UI t» 

X « 



* u. 

UJ M 

> *J 

UJ 

S3 

X 

• 

u> < 
O O 

> 5 

I? 

• C 

ui ui 

|M M 

Ui Mb 

-J M 

I* 

O D 

u a 
x 



>■ 4 

Si 



0 

o 

>- v 

A 

O ft 
X Q 

Ui Ui 
l~ X 
« 

ui i/i • 
ui 5 

X Xm 

t- UJ 14 

*5S 

A «C U 

Ui IA 
(Mm m 
iltO 

u ar 
ui <k ui 

Jlf 

ox a 
oo 
•J ^ 

< -i ui 
K o * 
4 U, 4 

I 

ui 



2 
* 

X 

s 

I 



i 



ui 
at 

3 



TAEG REPORT NO. 12-1 



01 



05 



iA 



0> 

ex 

•r" 
C 

x: 
u 
0* 



05 



in 



o 

ERIC 



Vl-214 



TAEG REPORT MO. 12-1 




TAEG REPORT NO. 12-1 





1~ 




o 








E 


<t5 






o 




<v 




> 




o 




u 






«J 


o 








U 






cn i/> 






s 


** 


c t> 

0) $ 




in 


>> o 






X) 


e 




«r- 




t/> 




*r m 


X> 


4) 


0> 


i- 


4-> 


3 




+-> 


U 


in 


3 






cn 






w 




CD 


< 


& 




O 








CD 


ire 


4) 


VI/ 

+-> 








OD-M 




<0 


in 


•r- 








cx 


AS 


o 




i- 




CJL 


"O 


a. 




rtJ 


CD 




c 


+-> 


' 1 — 


to 


N 


o 




E 


C 




(V 


in 


cn 


•r~ 






o 


+-> 




3 







Cn 
C 

o 

0) 

CJ3 

t> 

03 
O 

<U 
a> 

C 

o 



in 



-a c 

o o 

jc 

e a> 

<d ^ 

.c o 

</> fli 

5 ■ 
O 10 ^ 

jc: *o 
in 

2 O <M 
O in 
r- in cu 

*D 3 

o cr 

•r +J P 

jc a> a> 
h- E tn 



c: 
O 



ERIC 



VI-216 



TAE6 REPORT NO, 12-1 



« 



5 

: 



w 

is) 

-lis 

Ui I j 



I 

I 
I 



m j S N uS 
« I c 4 m u 

• ♦ * 4 5 4 

• I 

J 



• I O J J 

• > ! 

I 



11^4 1 

I it O I 

i o at t»* i 

I JUi4 iii | 

I UJ D X I 

I > «C ►» f 

I UiDU. < > 

■ T 

! ! 

i I 



2 

M 

i 

I- 

1 

i 



I 



*\ vu * 

*s yj 

• • -i ui 

* « At 4 X 

• 2 i- 

• • n «< 4 



i5 

» — — — 



-i 



9 

ERIC 



TAEG REPORT NO. 12-1 



o 
oo 

00 

8 



CO 

c 

•r- 
O 



O 

o 

o 

on 
c 
03 

g 

+-> 

00 





on 












03 




+-> 








O 




00 




<D 
















00 




■o 




i. 
















O «r- 




I*- r- 




(0 




+-> +-> 




SZ O 




o>+-> 








(0 +-> 




J- c 




4-> <D 




0> U 




<U 




03 jc 




O 




00 U 




«r— 




03 




03 




+-> O 




03 4-> 


03 


■o c 


^> 


' r— 


03 


<+- 


Q 


O - 




00 


<D 


00 C 


N 


;~ O 


>> 


00 «r- 


r— • 




'■0 




CI 


o3 > 


< 


c 




03 (U 




on 




oj xj 




x: o 




= 



9 

ERIC 



VI-218 



TAEG REPORT NO. 12-1 



UJ 



) fit 
) « 
r st < 
> * v* 

f a 



• * • • 



I uj 



< 

mi 

OA >- 



\ts » « 



• 1 
i 

U. I 



UJ O 
V- UJ 
4 M 
^ >• 
J J 4 
* « t« 
> F 4 
UJ * O 



Of 4 —-v-— • 



• 

I 
I 

I 

I 
I 

I 
I 
I 

-J 



i a 



* t 



* 

o o 
r m 

uj r -j 
> w a 
uj or f 

O M M 



4 

u 

VMM 

* * 
0*-i >• 
U> M -J 

t* * < 
a 4 



! 

i 

i 



-I UJ 



55 

uj x 



A. 

I MX ^ 

i co o u. uj 



4 



UJ 
* 

15 



o « 
** u. 

(3 UJ 

X wo 



o 



O K 



* # v 

UJ • 



UJ I 

* • 



ERIC 



TAEG REPORT NO. 12-1 



0> C to • 

C »r- i— Oi 

•r; C O C 

K -r- O O 

<u u u -o 

P4J W <l) 

<D N 

s: o i— 

O -M 0) 

P 1-0) 

O 0> "O 

to 3 c 

0) C ro 

<U 3 <TJ C 

x: ro 

0> 01 > i- 
0J 0> 

0 O 4-> 
<U O V- 

.c 

P a • * 

•r n o 
ecu 

• i/l 3 T 

"O C OP 
0)0*4-0 

(OP W t 

3 rtj (O d 

ro ai +j 

> u </i c 

01 o> 
a> to *~ 

«r- +-> -m 3 
u 

ro a> a> 

-M 4-> to flj 

ro ro <U 
tl E£ C 

««- -M O 

X> 4-> -r- 

0) to *~ +-> 

•r- ro C 

CO 3 

<0 4-> SZ <f- 

cn cn.n 

O C O to 

ro t~ 

>> -c a> 
-a * p x: 

a> to +-> 

i- c cn 

<U o c o> 

JC «r- «r- C 

3: +-> O 

ro Cn r— 

L0 3 r- 

0-+-> c ro 

0) r- O U 

4-> CO 

4/7 «M +J 

rO r «J C 

+-> 01 ro N ro 

ro X: 3 U 

O P 00 C U 

3 A1 ro 

x> oo c cn 3 

<U £ 3 t- 

N O O O 

>> X J*- +-> 

*— to o a» 

c e o cn 

*K O -V 3 

O >> o 

a» +-> o c: c 

4-> Ur- (O 01 

rd C 

3 3 O 0) 

If- P O W 

*J O 

> 3 * 0> r~ 

tiJ to oo to O 

TJ rd 

to C U 0) 

lO «#~ <U E 

c u o aj 

tO h« -M it U 



o 

ERIC 



VI -220 



TAE6 REPORT NO< 12-1 



i 
i 
i 



• — • 

i 
i 
i 



n m. 

M o 



4 VUJK 

Of X W 
J Q ^ 
i « U. * 
> * * Q 

UMMO 



(A I 





• 

1 




♦ 

t 




1 
1 
1 




i 
i 
i 




1 








1 


Of 


i 
i 




1 


n r > 


i 




1 


** o 






1 




K 




1 


« M> w* -1 






1 


m o r o 


i 




1 










<y Ul jn 


• 




1 


Ul M A X 






1 


l« M> O 


i 




1 


uj of rf 


i 




1 


a « o> r 


» 


« 


* 








• 




* 



■—X 



n 



I Nl In. 

I Ml < 

i of r 

i <o« 

| X M O 

I I J U. 

i r 

I tO > Ml 



* i 



-j 
m 

si 

M 

Ul OJ 
T * 4 
U O A 



I 

I UJ 

I o o 



, . 

I ! 

! ! 



3 



I o wo * 

• » T -» 4 

• I 

* • — — — — ft 



^ ♦ — — — — » 



• — 

I 
I 
I 

» 

I 

I 

I 

" uv 
t T 
1 



U. (.» 

4 r m> 

I Of WO 3P 



• — ♦ 

I 
\ 

I 
I 

i 

I 
I 



a \a a 

J H J Jf 

> M it 

uj uj a uj 
a a o i~ 



at ft « 



y 



t a or 

2 O 
4 v 



• I UJ 
* I O 

• I 
tf> ft 



« or 
& m 



u. » 

Ur | 
4 I 
I 

— ft 



• — — — 



^ UJ 

* Of o * 
m o y o 
I Ul m m 

^ I ft* T WO 
« | UJQtM M 
^ I **- UJ * > 

I uj uj or uj 
u. i a r m- * 

UJ I 

Of ft — — — — 



a 



*V wo 
\A uj 
# « 
o o 
mi * 

H» UJ 
UOk 
X m» wo 

~ Q - 
X * WO 
of et T 

UJ M> © 

wo? 
o o - 



» 

I 

I 

I 
I 



M o 
X WO 

* of a 
uj * * 

W* UJ UJ 

i I bu j y 

» I 

M> ft — ~ — ft 

A 



3 



UJ 

I 



3 

o 



• — — — — ft 



»•* UJ 

WA ft 3 V- ft 
ft 

^ • ^ < ^ ft 

• ^ r «i • 

U. ft li; * O ft 

'»» ft ft 

V ft ft ft ft • ft 



o n 
r *** 
< »• 

i n wo o 
i a o w 
• i u r j 

I > UJ Ol 

I UJ Of c 

I 6 H M 

W I 

r< * — — — - 



ERIC 



I wo 
I o 

t « 

I UJ 
I Of 



> w. 



I 
I 

I 
I 

mi t 



Of J» WO C \ 

Cl ^ Ol T « 

r m> m> i 

O «J > uj l 

tr» i U O BJ I I 

• I I 

UN ft — — — — ft 



TAEG REPORT NO. 12-1 



<D JC 
00 <P 
3 "O 

(X} O 
O <D -M <D 

x: 4-> -p ^ 

00 <13 00 

3 3 •#- «r- 

O E i- 

C Q. CXi 

<D C O C 

O t »r- 

(D'r aC 

•#— A3 <0 A3 <D 
4-> 0> J- "O 

•r- 00 i— <D 
^ (U 3£ 4/ 
<D > 4-> XI 

00 <f- 
C »r- O <S 

-r- a ov--> 

4-> <0 C 00 00 
OS X> »i ' <D 

o aj <n n <v 
<f- i- ctx: 

c 4-> u 

O <D *J r- 

> <D 4-> 3S M- 

•r* <7> 

Crt-M P JC 

flj flj O «r- 
UX££ C 

o +->+-> 00 <o 

c <£> 

00 O 00 

u a> x 3 

o e oo ej_ 

T» JC 

4-> r- HJ -p 

XT O r- 4-> 

U C 'r <0 P 

x: ka o 
^ a> >> <d 
o-ojcto 



*3 





x> x> 


n 




a 


o 




+-> 


r-" 


<v 


x: 


00 


00 


ft r— 


X> 


+-> 


•r- 




<D <D 


•O 




x: 


<u 


00 cx 


<D 




h- 


•C 


03 00 


Fe 








u 








4-> 




CD 


< 


oo 


<Q SZ U 


C 




to x: 


-M A3 






<u 




<D 


c 




r— 




+J r- 


•r- 


x: 


<U 


O O 




u 


+-> 




l- 




•■o 




jO 


O 




-a 


o 


•r- 


00 00 




TO 


</J 


■r- 








00 


<U 


O 




(/> 


c 


00 X! 








CL 


x: 4-> 


■o 


4- 


C 


UO 


4-> 00 


<u 


o 


o 


•r- 








•r- 










(/I 














00 






> 




*> T 


c 


u 


re 




U XT 


E 


GJ 






> 




x: 






<D 


+j 


LO 


c 


3: • 


+-> 








o -o 


<D 


oo 








O 




CO 


O 


> 

r— 1 




lO 




A3 


• o 


«X> 


*i — 






XJ > 




X 


o 


O 


o c 


ID 


h— 


4J 





9 

ERIC 



VI-222 



TAEG REPORT NO. 12-1 




• # 

# • Ui 
« M > 
• *4 4 • 

uu o 

• ♦ 



i 
i 
i 



* H 13 * 

~ O * O 
K ~« 

I • UJ ft *- M * 

« • fc* UJ 4 > • 

# uv WJ V w # 

u • O Jt « • 

u* # • 



I 



UJ I 



I IU * => t* I 
I f 4 JM I 

I mj5 I 

I * 4 u x f 

4 « M UJ I 

f UJ * U m 4 

I MUJ V> I 

I UJ X D UJ I 

I Q "B Li at | 

I I 



OS* • 
t 4 N. • 



I 
4 



LU < UJ 



« i * 

vs * — — # 



I- O 

JUiM 



> t 
UJ I 
LU yU C* I 



*^ • — — — — • 



• # 

# • 

>» o 

■■ ♦ M N 

# Sr • 

* 15 * 

M 

• M • 

• 



t I 
I 
I 



Ui — 

r u, o 

« V MM 

to » * w r u i 

»f> i m * u> t 

I tu < or i 

U> I O i h UI | 

UJ I j 
* « _ — — — ♦ 



T ~~~~ * 



I UJ 

I U Q 

— < 5 t£ 

I h m 3 O 

I 3+* ui vi 

i jse<^ 

m I 413 > 

• f > 4L Us 

£ 1 Ui <A O « 



u. 

UI 



OUIJ 
*i sA m 

> a »~ 

UJ © UI 
OUfi 



o 



5 

M 

> 

M 
* 

3 



o 

UJ 



! 



ss 



lA I UJ UJ M j 

*I5S2 I 

m ; 



ERIC 



TAEG REPORT NO. 12-1 



Summary of Major Technological Gaps and Problem Areas - 
Function 5.0 Evaluate Performance (Feedback) 

1) There 1s apparently no clear understanding at the school 
house level of the high significance of feedback. As a 
result, a number of areas Ignore 1t totally. 

2) Where serious attempts are being made to contact school 
graduates or their supervisors concerning feedback, the 
method 1s almost exclusively a mailed questionnaire. 
The time frame 1s usually six months after the student 
leaves the course. 

The rate of return for such questionnaires 1s very low. 
The highest return found was thirty-eight percent and this 
was given to graduates of an Instructor Training course 
located in the Immediate vicinity of the school. When 
forms are to be obtained from ships at sea, the rate of 
return drops drastically. One training locality was 
enthusiastic about going as high as twenty percent under 
such circumstances. While no overall figures could be 
obtained, for the average rate of return 1 t 1s probably 
ten percent. Furthermore, 1t 1s doubtful those that do 
take the time to fill out the form are representative of 
the total group. 

Regardless of the rate of return, the questionnaires 
themselves leave much to be desired. Specifics are 
almost never addressed. Questions are so broad that 
almost any kind of Interpretation may be used. In 
short, the forms are not appropriate vehicles to elicit 
Information. 

A further problem for "A" school graduates concerns the 
usual six-month time period. At best, the graduate may 
have had three months in his job role, since the usual 
experience in reporting on a ship is an Initial three- 
month period as mess cook. If there are travel delays, 
leave, or further schooling, the delay will be even 
longer. It is probable that when the individual receives 
the question form that he has had three months or less on 
his job. It is not likely that he can evaluate his train- 
ing meaningfully in that time. 

3) Some schools utilize new people reporting from the Fleet 
for feedback information. The question forms are again 
deficient. In addition, there was no apparent recognition 
of the limited sampling of personnel, equipments, and 
ship types. The technological gap here is not in the use 
of such personnel for feedback, but rather in the way the 
deta is obtained. 
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4) There 1s no Integrated effort to obtain feedback but 
rather a series of Isolated and unrelated Independent 
approaches. This 1s somewhat understandable where there 
1s just one school. However, 1n a number of Instances, 
there are two or three schools, geographically separated, 
but teaching the same courses. No collaborative efforts 
to obtain feedback were found. 

5) Several notable approaches were found such as the "Ship 
Riding" program of the EW "A" school , Naval Schools 
Command, Treasure Island. Apparently, the details of 
the approach were not known by other schools or commands, 
or 1f known, were certainly not Implemented. 

This Is further exemplified by the research efforts of 
NPTRL. This organization {now called NPRDC) has conducted 
a number of excellent investigations dealing with approaches 
to feedback. Again, there has been little or no impact on 
approaches to feedback. The gap is 1n the Inadequate dis- 
semination of information on new feedback approaches. 
Additional possible related problems are the apparent 
unwillingness of training personnel to try new techniques, 
or failure to provide resources for such approaches. 

Idealized Approaches - Function 5.0 Evaluate Performance 
(Feedback) 

One approach to Idealization 1s as shown in the flow charts. 
Here a systematic evaluation of available methodologies, 
planning, and careful preparation is stressed. While a number 
of methods may be examined, 1t must be emphasized that each 
method must be considered 1n its most, rather than least, 
effective application. 

To amplify this point each potential approach listed 1n Table 
VI 5.1 will be considered: 

1) Experimental Method - This must be a carefully controlled 
experiment, properly designed and conducted 1n a laboratory 
(or near laboratory) context. It will necessitate a pro- 
fessional approach and a complete evaluation of the 
statistically stated hypotheses. It will ordinarily re- 
quire from 3 to 6 months after the preparatory phases are 
completed. However, complex Investigations may well re- 
quire a year or longer. Preparation depends upon the 
degree of equipment and material involvement. This may 
take from 1 to 6 months. Subjects for the study must be 
those that fit into an operational situation. For the 
time they participate in the study, they are ordinarily 
unavailable for other duties. 
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EVALUATION - A restricted research approach which 
1s highly valuable but 1s not amenable to applica- 
tion by non-professional personnel. Properly used, 
1t will yield extremely valuable performance feed- 
back information. Generally this should be a 
research project dealing with training (schools) 
Involving large numbers of people, or that has 
special problems. 

Mall-out Questionnai re - If this approach 1s desired, the 
questions must be carefully constructed to elicit the 
kind of Information that could be useful 1n revising a 
course. This means completeness and thoroughness which, 
further, automatically means a lengthy and perhaps com- 
plicated form. Anything less, yields superficiality. 
Unfortunately, doing 1t effectively guarantees a lower 
rate of return than 1s obtained currently. 

EVALUATION - This method has app al since 1t has 
seemed to be relatively easy to use. However, the 
information obtained has been minimal. In order 
to get adequate Information, the form must be made 
in a detailed manner. This means that few such 
forms will be returned because of increased com- 
plexity and time to fill out. The mall-out ques- 
tionnaire approach must be seriously questioned and 
then probably dropped. 

Performance Diary - The Performance Diary demands a very 
high degree of cooperation from the technician. In 
addition, it requires much of his time and effort. An 
effective diary necessitates an effective welter also. 
There is a question as to how representative such 
technicians would be of their peer group. On the other 
hand, information from this kind of source would be an 
Invaluable font of insight. The key element here 1s the 
highly capable technician who can also write 1n a ludd 
manner and who is motivated to gather this data. 

EVALUATION - If personnel can be found to keep such 
diaries, the information contained therein would be 
very helpful. The odds that such people are around 
are slim. Moreover, the data obtained from such 
sources must be carefully evaluated. The fact that 
a capable technician feels that a part of a course 
helped him doesn't mean that it helped the average 
student. On the other hand if the "diarist" reports 
that he has difficulty, or cannot perform an opera- 
tion which is taught in the school, a close examina- 
tion of the instruction should be made. 



VI-226 



TAEG REPORT NO, 12-1 



If appropriate personnel can be found, they should 
be trained 1n the keeping of a performance diary. 
The voluntary nature of such participation must be 
emphasized, as well as the value of their efforts. 
Data so obtained should be carefully evaluated. 

The performance diary would be of great help as a 
source of feedback. All that 1s required 1s the 
right kind of diary keeper. 

Analysis of Existing Records - Data from records such as 
3M, Maintenance Data Forms, Records of Mobile Technical 
Units, Fleet Training Group Reports, etc., are primarily 
hardware oriented. It 1s nearly Impossible to relate hard- 
ware problems to the training that the Individual has had. 
On the other hand, a picture may be obtained as to the 
most common types of problems. These can be compared to 
the training elements. Certainly those problems occurring 
most often should be treated specifically during training. 
One problem with these records 1s that they are not dis- 
tributed widely. Still, a school should be able to get 
what 1t needs by request made at the proper location. 

EVALUATION - To the extent that hardware 1s Involved 
1n the training, these records will Indicate sig- 
nificant points to be covered 1n training. Some of 
the data might give hints/clues as to potential train- 
ing problems. The worth of the data may be suspect 
1f data Inputs are limited or skimpy. 

This method should be used whenever possible. The 
data must be rngarded as Indicative and should be 
verified by cross checking of other sources. 

Rotation of Fleet/Training Personnel - Obtaining informa- 
tion about the adequacy of training from newly reporting 
staff personnel has great appeal. They are presumably 
capable, well motivated, and knowledgeable people. 
Certainly they could tell how effectively people from 
schools can operate 1n their jobs. Within limits, these 
are reasonable surmises. The limits involve the scope of 
the Individual's experience. To what extent did he com 
in contact with recent school graduates? What was the 
nature of these contacts? In what capacity? How recent? 
How many people were involved? What were the specific 
instances? What kind(s) of equipment? 



VI -227 



TAEG REPORT NO. 12-1 



The only method observed of obtaining Fleet feedback from 
these people was by means of a question form, The form 
asked very few questions and these were of a general nature. 
None of these forms could be considered as meeting even 
minimal requirements. 

EVALUATION - Elldtatlon of Information from new 
personnel arriving from the Fleet could be extremely 
valuable. Such Information must be obtained by 
skilled debriefing. This should take place very 
shortly after arrival at the training organization. 
Obtained information should be evaluated In the light 
of the respondents experience and qualifications. 
All opinions must be backed by his observations and 
preferably by critical Incidents that give specific 
examples. Cross checks are important to validate 
Information received. Properly performed, the method 
of getting feedback information from new personnel Is 
highly valuable. If all precautions are not followed, 
the data may be misleading. 

6) Informal Feedback - Such feedback may be by letter, phone 
call, personaT contact, or just 1n passing. Most of this 
kind of input is broad and could be nearly meaningless. To 
place structure on such data, follow-up discussion should 
try to place meat on the bones. If the discussant can't 

be specific about "that lousy training" he may have 
seriously overstated the issue. If he can give details, 
significant information may be obtained. 

EVALUATION - without elaboration, broad Informal 
comments are not helpful. With more details, sig- 
nificant data may emerge. Such data should be con- 
sidered as a starting point for further investigation, 
rather than taken directly at face value. 

7) Informal, Visits - Unstructured - This kind of visit 1n- 
volves~generaT~dTscuYsion rather than a search for 
specific points. However, with skill and some digging, 
the broad areas can be directed into specific points 
pertinent to training. Such visits do not entail forms, 
checklists, or other pertinent kinds of preparation. 
Note taking may or may not be used. Usually it will be 
difficult to cover very many people under such circum- 
stances. 

EVALUATION - This approach may be helpful if the 
right people are contacted and proper questioning 
techniques are used. It is difficult to collate 
inputs from different personnel but not very many 
people can be covered anyway. Also some kind of 
crosscheck of responses is possible by following 
up leads gathered in earlier contacts. 
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For the most part, this approach 1s not as fruitful 
as the one to be covered next (Visits - Structured 
Interview). However, 1t does. have some advantages. 
First, 1t requires little or no preparation. Second, 
1t does not necessarily require a high degree of 
technical knowledge. Third, 1t has the distinct 
advantage of demonstrating to Fleet units that the 
training system cares enough about supporting them 
to come aboard. 

If for some reason a training unit cannot perform 
structured Interviews, the Informal visit 1s worth- 
while from a "public relations" point of view. 
Properly performed, 1t could extract good feedback 
Information. 

8) Visits - S tructured Interview - This approach 1s described 
1n"detainn NPTRL Research Report SRR 72-13 (Jan. 1972). 
It entails the use of a structured question form based on 
the job of the Individual. The details of the Job are 
structured 1n terms of the task analysis. Forms are made 
to be completed in about one hour. They are administered 
on a face-to-face basis. There 1s no problem about re- 
turns or waiting months for replies. The form 1s completed 
then and there. Experimental applications of this technique 
(summarized 1n the report) have yielded excellent results 
which were directly applicable to the training courses. 
A variation of the technique applied to the Storekeeper 
rating also yielded highly useful feedback (SRR 72-14, 
Jan. 1972). 

A specific outgrowth of this technique 1s the system pro- • 
posed by the Propulsion Engineering School, Service School 
Command, Great Lakes. Card decks are prepared pertinent 
to four different jobs for BT's and MM's. Each card 
contains task statements obtained from the task analysis. 
An Interview 1s conducted by a senior petty officer 
technically qualified in MM/BT operations, and 1s given 
to one person at a time. The respondent reads each card 
and places the card 1n three categories: 

"Don't Do" 

"Do With Difficulty" 
"Do With Ease" 

After all cards are sorted, the interviewer asks the reasons 
that cards have been placed in the first two categories. 

All recording is done by the Interviewer. After the 
interview, the immediate supervisor is asked about the 
individual's capabilities in four areas: 
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Watch Standing 
PMS Checks 
System Line-ups 
PQS Progress 

This approach 1s essentially a type of structured Inter- 
view also. 

The Ship Rider Program of the EW "A" School seems to be 
something between an Informal and a structured program. 
It 1s much closer to the structured situation, since 
detailed observation and close contact Ts Involved. A 
senior technician spends some time at sea aboard repre- 
sentative ships. Methods used vary from observation, 
Interview, Informal conversation, and review of equip- 
ment/system developments. A significant factor 1n these 
visits was that Fleet personnel were given a more de- 
tailed view of what the school was teaching and the 
school observers determined which publications and guides 
the users were employing. Specific useful Information 
was gained and course changes have been Instituted as a 
result. 



EVALUATION - This approach is the only one found that 
yields consistently useful Information. It possesses 
the advantages of the informal visit, but comes up 
with much more significant information. By using the 
existing task analysis, much of the work of prepara- 
tion is reduced. Information 1s required, however, 
about frequency of operations/maintenance as well as 
about which tasks are performed other than those per- 
taining to the NEC or rating. 

Visits should also determine documents, local policies, 
and tactics related to the utilization of the personnel 
of primary concern. Such additional effort may not 
only be most helpful to the school program, but also 
will further show ship personnel that the school wishes 
to assist them in a positive manner. Also observa- 
tions may be made which will lead to the development of 
effective training and performance aids. 

A certain amount of time and travel is required for 
such visits. This is well merited and is not excessive 
when considering the "effectiveness" along with the 
cost. Very cheap methods give very cheap results. 
Cost-effectiveness is then low. In the event that the 
required result is not achieved, the cost-effective- 
ness falls to zero. If somewhat more expensive costs 
give excellent results, the cost-effectiveness is 
liigh, 0 nd more feedback for the funds has been 
achieved. 
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The preceding section was a capsule analysis of the 
potential common methods for obtaining feedback. The 
"perfect" kind of feedback 1s the PTEP, an FBM approach 
listed as a footnote 1n Table VI 5.1. This program 
establishes a set of standards for essential ratings/ 
jobs, and develops tests to determine how well the 
Individual has achieved the standard. Each person In a 
position covered by tests is evaluated when he 1s in the 
off-patrol status. Results Include a description of what 
additional training/study Is necessary to bring the Indi- 
vidual up to standard. The PTEP 1s used 1n the FBM 
community alone. If all areas of the Navy had circum- 
stances similar to FBM's, this kind of approach Is indeed 
the ultimate. However, only the FBM force has the re- 
sources of the Strategic Systems Project Office (SSPO) ; 
Blue and Gold crews; and a relatively small number of 
people. Consequently, a PTEP-like approach 1s not con- 
sidered to be practical at this time for the remaining 
naval forces. However, Individual training organizations 
or ships could well establish, on a limited basis, their 
own kind of evaluation program, with the goal of deter- 
mining training needs. This would provide outstanding 
training feedback to the training system. In considering 
potential approaches, training units should consider the 
PTEP approach also to determine if the total approach, or 
parts might be feasible for their application. 

The other kinds of approaches mentioned at the bottom of 
Table VI 5.1 might be considered as feasible and appli- 
cable for each training area. All of the listed methods 
offer significant potential. The most important point 
to be made is that a number of techniques should be used, 
rather than only one or two. 

The approaches given represent one step toward Idealizing 
the feedback function. They represent large steps for- 
ward, but idealization cannot stop there. A total new 
system approach must be developed concurrently, one that, 
if started soon, can bear fruit 1n the 1980 time frame. 
This is both an extension of the first methods and a new 
point of departure. 

There are several fundamental philosophies in this "Neo- 
Feedback" gestation: 

o Involve students in the total training-operating 
context so as to make them active and involved 
partners. This is an Initial step in turning 1 
them Into professionals and an essential start. 1n 
getting the best feedback from them once they are 
on the job. 
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o Involve Fleet personnel as active participants In 
the total naval situation. While the students of 
today are the leaders of tomorrow, that day Is some 
years away. Until "tomorrow," the supervisors of 
today must be considered. 

o Enlarge the span of operations for a "school" from 
a strictly Instructional unit, to one that Includes 
active assistance 1n operational and training problems 
of Fleet units. Such Involvement would be through 
CHNAVTRASUPP, who Is responsible for supporting all 
training. The assumption here 1s that training and 
operations are not clearly separable activities. 

o Operate major feedback operations on a cooperative 
basis through upper command levels such as Service 
School Commands or higher. An example of a major 
effort would be a ship rider program and/or struc- 
tured visits. 

Each of these points will be expanded 1n the following 
paragraphs. In addition, a general way of achieving 
these goals will be presented. 

Student Involvement - The theme must be played con- 
stantly that the student 1s a significant member of 
the Navy community, that he 1s needed, and that he has 
a participatory role. This kind of Indoctrination can 
be presented 1n orientations, Interspersed throughout 
self Instructional materials, but most of all through 
the treatment he gets and his chance to actively 
participate, in the total process. Wide spread use of 
Individualized instruction 1n Itself Increases parti- 
cipation. In addition, the idea that students can let 
the school know how well the training has suited them 
for their jobs should be stressed throughout *he1r 
training and reemphasized at graduation. Another 
aspect that should be shown is that some of them might 
return to the school as students 1n advanced classes 
and finally as staff personnel. 

The essential objective 1s to attune the student to 
relate his training to his job and to evaluate his 
training for that job. Also, to originate the ex- 
pectation that at a later date he will be contacted for 
information about his job and the training that he had 
received. 

A second objective is to show that there are open 
communication lines to the school and to encourage 
their use. This premise will be developed shortly. 
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The achievement of these objectives will plant seeds 
that may come to fruition some time later in the 
form of realistic, appropriate feedback Information. 

Involve Fleet Personnel - To Involve Fleet personnel 
will require changing their attitude that they are 
outside "training." In order to accomplish this 
thoroughly, a total turnabout 1s necessary. Ulti- 
mately, the scene should change so a tour 1n training 
will mean to an officer what a ship command means now. 
However, this kind of drastic (but feasible and needed) 
change 1s not fully necessary Immediately, to get 
more Involvement of Fleet personnel. What will help 
1s the opening of effective lines of communication as 
mentioned previously, and to be explained 1n the next 
paragraph. 

Enlarged Span of Operations for a School - The school 
must realize that 1t is the primary seat of knowledge 
for Its area of specialty. It will often generate 
excellent outlines, performance aids, references, etc., 
which would be Invaluable on the job. Probably the 
most skilled technicians/operators in the specialty 
field are on the staff. Consequently, 1t 1s necessary 
for the school of the future to serve as an active 
direct aid to the Fleet, 1n addition to training 
personnel . 

In part, this means developing effective communication 
channels. For example, a phone number for answering 
questions about training or other related technical 
matters may be established. Such a number should be 
made available to graduates, and officers/supervisors 
related to that specialty area. Currently, there 1s 
at least one instance of such a channel. This 1s for 
the Engineering Systems Training Division, Training 
Department, Fleet Training Center, San Diego. In giving 
their number they state: 

"This line can be used at any time to relay 
Information of training nature to us or to 
request our assistance in obtaining informa- 
tion of a technical nautre." 

This is one type of communication that is meant. Of 
course an FTC has the advantage of proximity to the 
Fleet. Thus it is necessary to encourage messages and 
letters as well for training areas more remote to 
Fleet ports. Encouragement through active solicita- 
tion is essential. However, the key is the kind of 
interaction that occurs. Requests for aid should be 
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cared for promptly and appropriately. The quality 
and timeliness of help 1s all essential. Comments 
on training should be explored for details as much 
as possible, and accepted with thanks regardless of 
criticism direct or Implied. 

One type of solicitation which will encourage com- 
munication to a high degree 1s to obtain the names 
and addresses of oil Individuals newly promoted 1n 
a particular rating, and send them letters of con- 
gratulations. Such letters should reiterate the 
Individual's significance and his Importance as a 
participant In the naval training loop. In addition, 
1t should encourage his use of the open communication 
channels. 

Another method of keeping such communication going 1s 
to maintain an active file of all personnel names, 
addresses, and assignments 1n that particular 
rating/NEC. An Intermittent newsletter of technical 
Information, requests for comments on training 
adequacy/needs, or new a1ds/charts/t1ps could be sent 
to such personnel. This kind of approach will further 
bind the Identification of the Individual to the 
school. Of course the keeping of such data 1s con- 
tingent on effective computerization and information 
retrieval. Smaller schools can, 1f necessary, main- 
tain such records manually. 

Cooperative Feedbac k - Currently, feedback approaches 
(when an effort 1s made) come from each school. No 
attempt 1s made to relate to "counterpart" schools or 
to related schools. It 1s apparent that duplication 
occurs and Information 1s not obtained that should be. 
It is necessary to organize team feedback approaches. 
These could be of several different types: 

o For schools that are located at more than one loca- 
tion - One "task force" group representing all 
locations could be organized for structured visits. 
Training of the team and the planning for visits 
should be handled by a higher command level such 
as CNET, CNTECHTRA, or a Service School Command/ 
Naval Training Center. This would cover one rating 
across the board and Include graduates of "A" and 
"C" schools. 

o For a number of different kinds of schools - A 
Service School Command contains a variety of 
schools from quite large to relatively small. 
Coordinated visits from a number of different 
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schools under the control of the school command 
have a number of compelling features. Training 
and preparation for the visits will be uniform. 

The amount of time spent on administrative planning 
will be reduced, since duplication 1s eliminated. 
To an extent, the burden on the ships will be re- 
duced also, since they would deal with a central 
authority, rather than a number of segments. The 
quality of the feedback will probably be enhanced, 
because of the opportunity for various individuals 
to exchange their experiences and methods, as well 
as because of the tighter management controls. 

The fundamental assumption Inherent 1n the foregoing presenta- 
tion 1s that feedback 1s all Important. Without adequate In- 
formation about how well the training prepares personnel for 
their later job, there Is a serious danger that the training 
may be deficient. Mention must be made of the fact that some 
students go directly to advanced training after "A" school. 
There Is further need for feedback here 1n terms of how well 
the "A" school prepares personnel for later training. Such 
Information must be obtained by direct technical liaison of 
training directors and/or Instructor personnel. 

The methods most emphasized as effective tend to Involve travel. 
It 1s recognized that this 1s a problen, currently because of 
travel restrictions. It 1s hoped that a close look at poten- 
tial cost benefits might relieve this restriction to an extent. 
Use of cooperative approaches with the utilization of naval 
aircraft may make travel costs minimal. Also, highly effec- 
tive annual feedback efforts are worth much more than poor 
efforts conducted more frequently. 
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Function 6.0 Manage Training Resources 
a. Definition 

Management of resources can be viewed as a separate subsystor , 
that 1s 1n contrast with the technical subsystem. The two sub- 
systems, however, are Intimately related and interact 1n many 
ways. Both subsystems are driven by requirements both Interne' 
and external to the NETS. 

One of the subfunctlons of resource management, "Formulatt 
Strategic and Fiscal Guidance," while shown Integral to the 
"Manage Resources" function drives both the technical as well 
as the resource subsystems. Its translation and retranslate 
at various command levels, however, 1s vital to the development 
of effective guidance for maintaining a viable resource manage- 
ment function. 

The overall resource management function Involves the planning 
(development of guidance), programming (development of program 
objectives), budgeting, and control of resources. Since re- 
source needs must be estimated based upon some technical plan, 
the t1e-1ns to the technical subsystem are through the 
"Analyze and Plan Training" function as an input, and through 
the "Implement Training" function as an output. 

A major subfunction of resource management observed at all 
command levels 1s "Estimate Resource Requirements." This sub- 
function performs the quantification of any technical plan for 
Introduction Into the programming, budgeting, and controlling 
of resources. While the names of the other subfunctlons 
within resource management were chosen to roughly coincide 
with definitions under the Planning, Programming, and Budgeting 
System (PPBS), some license was taken to obtain more loqlcal 
groupings at the sub-subfunct1onal levels. Also, to prevent 
repetition the estimation of resource requirements was broken 
out as a separate subfunction. 

Any system to remain effective must have some method of evalua- 
ting Itself. Therefore, a separate subfunction, "Monitor, 
Analyze, Evaluate, and Improve Function of Resource Management" 
was identified. 

The major subfunctlons of resource management and the more 
apparent ways 1n which they interact "are shown 1n Figure VI 
6.0 - Manage Training Resources. Several strategic document:,, 
periodic studies and analyses, policies, and schedules provide 
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the more significant Inputs to the system. A major objective 
1s to have training plans quantified and Incorporated Into the 
resource management function 1n as accurate and timely manner 
as possible so that training implementation can proceed In an 
optimal manner without undue restriction from the resource 
management function. It 1s toward this end that functional 
Improvements should be directed. 

Description 

The Manage Training Resources function 1s described through an 
outline which contains all of the functional elements pre- 
sented 1n the functional flows, and In addition through an 
overall functional flow, Figure VI 6.0 - Manage Training 
Resources. 

Page 

6.0 Manage Training Resources 249 

6.1 Formulate Strategic and Fiscal Guidance 252 

6.1.1 Develop Joint Intelligence Estimate Plan 
(OIEP) 

6.1.2 Perform Joint Long-Range Strategic Study 
(JLRSS) 

6.1.3 Develop Joint Research and Development 
Objectives Document (JRDOD) 

6.1.4 Develop SECDEF Planning Guidance 254 

6.1.4.1 Develop Material Support Planning Guidance 

6.1.4.2 Develop Defense Policy and Planning 
Guidance (DPPG) 

6.1.4.3 Review of Defense and Material Support 
Guidance Throughout DOD 

6.1.4.4 Finalize Planning Guidance 

6.1.4.5 Develop Strategy Guidance 

6.1.4.6 Prepare Schedule for Annual PPBS Cycle 

6.1.4.7 Prepare and Issue Planning - Program 
Guidance Memorandum (PPGM) 

6.1.5 Develop Joint Forces Memorandum (JFM) 

6.1.6 Develop Navy Planning Guidance 256 

6.1.6.1 Develop CNO Views on Strategic Objectives 

6.1.6.2 Develop Specific CNO Objectives 1n Support 
of SECDEF Objectives 

6.1.6.3 Develop Broad Guidance for POM Development 

6.1.6.4 Issue CNO Policy and Planning Guidance 
Document 

6.1.7 Develop JSOP Volume II - "Analysis and Force 
Validation" 

6.2 Develop Program Objectives Memoranda 258 

6.2.1 Develop and Maintain List of Training Re- 
quirements in Priority Order 260 
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6.2.1.1 Transmit Information Through Command 
Levels to Cognizant Person at CNET 

6.2.1.2 PrQpara Training Requirements Summary 
CNT Form 1500/3 

6.2.1.3 Assign Plan Control Number and Enter 
1n Control Log 

6.2.1.4 Distribute Training Requirements Summary 
to Committee Members for Evaluation 

6.2.1.5 Determine Capability to Meet Training 
Requirements 

6.2.1.6 Review by Training Requirements Committee 
and Assignment of Priority 

6.2.1.7 Training Requirements Committee Recommends 
Implementation 

6.2.1.8 Recommend No Implementation to Training 
Priorities Board 

6.2.1.9 Identify Resource Deficiencies and Lower 
Priority Compensation 

6.2.1.10 Review by Training Priorities Board for 
Decisions 

6.2.1.11 Place Requirement on "Add-On" List for 
Access in POM Cycle 

6.2.2 Call for Navy Education and Training 262 

6.2.2.1 Develop Specific Guidance Related to 
Differences in Functional Commands 

6.2.2.2 Issue Guidance and Request POM Cycle 
Inputs 

6.2.2.3 Aggregate and Prioritize Requirements 

6.2.2.4 Analyze and Reprloritize Total Requirements 

6.2.2.5 Submit Requirements Through Commands to 
Major Mission Sponsor (ONET). 

6.2.3 Develop Navy Education and Training Tentative 

POM (TPOM) 264 

6.2.3.1 Review and Constrain TPOM Input - 
Prioritize Addendum (DNET) 

6.2.3.2 Input Resource Rtquirements to Navy 
Resource Model (NARM) 

6.2.3.3 Identify Resource Shortfalls as Compared 
to Prior FYDP Update 

6.2.3.4 Develop CNO Program Analysis Memoranda 
(CPAM's) (0P9O1) 

6.2.3.5 Develoo CPAM's with Identified Alternative 
Actions '(0P901) 

6.2.3.6 Assess Alternatives and Submit Impact 
Statements (DNET) 

6.2.3.7 Working Group Assesses CPAM's and Impact 
Statements 

6.2.3.8 Steering Committee Assesses CPAM's and 
Impact Statements 
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6.2.4 Obtain Initial CNO Approval 266 

6.2.4.1 CNO Executive Board (CEB) Reviews 
CPAM's and Recommendations 

6.2.4.2 CEB Makes Tentative Decision on Each CPAM 

6.2.4.3 SECNAV Amplification of SECDEF Guidance 

6.2.4.4 Summarize CPAM's Identifying Over/Under 
Control From SECDEF Guidance 

6.2.4.5 Sponsors (DNET) Assess Decisions and 
Develop Impact Statements 

6.2.4.6 Working Group Reviews Impacts and Makes 
Recommendations 

6.2.4.7 Steering Committee Reviews Impacts and 
Makes Recommendations 

6.;\4.8 CEB Reviews and Recommends Final Decisions 

6.2.5 Complete Approval Cycle Through CNO, SECNAV, 

and SECDEF 268 

6.2.5.1 Reassess POM Considering Impact Statements 

6.2.5.2 CNO Final Decisions and Approval of POM 

6.2.5.3 Submit POM to SECNAV for Approval 

6.2.5.4 Review of POM by SECNAV and Identification 
of Over-Controls 

6.2.5.5 SECNAV Final Decisions and Approval of POM 

6.2.5.6 Submit POM to OSD for Approval 

6.2.5.7 Prepare Issue Papers for Review and Comment 

6.2.5.8 Review of Issue Papers by Major Mission and 
Appropriation Sponsors 

6.2.5.9 Assessment of Issue Papers by OSD 

6.2.5.10 Issue Program Decision Memos (PDM's) to 
Spon. f nrs 

6.2.5.11 Prepare Impact Statements on PDM's 

6.2.5.12 Submit Impact Statements to NAVCOMPT for 
Budget Markups 

6.3 Formulate Budget 270 

6.3.1 Call for Navy Education and Training Budget 

Estimates 272 

6.3.1.1 Mod 1 f > Prior Guidance as Required for 
Budget Estimate Cycle 

6.3.1.2 Detail Specific Requirements for Forms and 
Formats of Input 

6.3.1.3 Provide Control Figures by Resource Category 
as Required 

6.3.1.4 Provide Schedule for Inputting Budget 

6.3.1.5 Provide Any Background Information That 
Seems Relevant 

6.3.1.6 Compile Information and Issue Necessary 
Instructions 

6.3.1.7 Retranslate Instructions at Intermediate 
Levels of Command 
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6.3.2 Prepare Oudget Documents and Supporting 

Data 274 

6,3.2.1 Develop O&MN Budget Estimate 276 

6.3.2.1.1 Develop Civilian Personnel Cost 
Analysis by Program Element 

6.3.2.1.2 Develop Costs for Aircraft Operations 
and Flight Training 

6.3.2.1.3 Develop Training and Cost Data for 
Serv1ce-W1de Training and Education 
Reports 

6.3.2.1.4 Develop Initial IMRL Requirements 

6.3.2.1.5 Develop Other Costs Including 
Relmbursables 

6.3.2.1.6 Develop Facilities Management Costs 
(Identify Unfunded Requirements) 

6.3.2.1.7 Develop Budget Submission - (CNT 
Form 7110/1) Per Guidance Figures 

6.3.2.1.8 Develop Unfunded Requirements - (CNT 
Form 7130-18) 

6.3.2.2 Develop OPN Budget Estimate 278 

6.3.2.2.1 Develop ADP Equipment Requirements - 
(CNT Form 7043/1) 

6.3.2.2.2 Develop IPE (Industrial Plant Equip- 
ment) Requirements - (CNT Form 7043/1) 

6.3.2.2.3 Develop Film Requirements for 
NAVTRASUPCOM on Request 

6.3.2.2.4 Develop Initial Spare Parts Require- 
ments - (CNT Form 7130/13) 

6.3.2.2.5 Develop Training Equipment/Devices 
Modification Requirements (CNT Form 
7043/1) 

6.3.2.2.6 Develop Other Legitimate OPN Require- 
ments Costing Over $1,000 

6.3.2.2.7 Prepare Detailed Justification 
(Different Requirement for ADP Equip- 
ment) *>» 

6.3.2.3 Develop RDT&E Budget Estimate 

6.3.2.4 Develop MILCON Budget Estimate 

6.3.2.5 Develop Inputs to Other Appropriation 
Sponsors as Required 

6.3.2.6 Develop Ancillary Data for Local, 
Functional, etc., Report/Control Systems 

6.3.2.7 Submit Budget Information to Required 
Department 

6.3.3 Review Budget Submission and Aggregate at Each 
Command Level - 280 

6.3.3.1 Review Budget Submission for Technical 
Accuracy 

6.3.3.2 Review 8udget Submission for Reasonable- 
ness by Category 
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6.3.3.3 Review Supporting Data for Adequacy of 
Justification 

6.3.3.4 Negotiate With Originator to Obtain Viable 
Budget 

6.3.3.5 Analyze Unfunded Requirements and the 
Priorities Assigned 
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When Required 
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Budget to NAVCOMPT 

6.3.4 Conduct Hearings and Markup to Obtain Final 
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Claimants for Assessments 
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6.3.4.6 Submit Budget to SECNAV for Approval 

6.3.4.7 Submit Budget to OSD for Review and 
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6.3.4.8 Conduct Hearings at OSD to Identify Re- 
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6.3.4.9 Prepare Program Budget Decisions (PBD's) - 
Transmit to Sponsors 
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6.3.4.13 Finalize OSD Education and Training Budget 
(Wi thin Appropriation) 

6.3.5 Submit Budget to Congress for Markup and 
Approval 284 

6.3.5.1 Prepare President's Budget and Submit to 
Congress 

6.3.5.2 Review by House Committees - Appearance of 

Witnesses 



VI-242 



TAEG REPORT NO. 12-1 



Ems 



6.3.5.4 

6 • 3 . 5 . 5 

6 . 3 . 5 1 6 
6.3.5.7 



6.3.6 



6 . 3 > 6 • 2 
6.3.6.3 
6.4 Execute 



6.3.5.3 Identification of Changes, Decrements, etc., 
to NAVCOMPT for Budget Markup 
Review by Senate Committees - Appearance 
of Witnesses 

Identification of Changes, Decrements, etc., 
to NAVCOMPT for Budget Markup 
Prepare to Authorize and Appropriate Funds 
OSD Allocation of Funds for Continuing 
Operations 

Appropriate Funds to Permit Operation 1n ' 

Fiscal Year 2,v 

6.3.6.1 Congress Authorizes PAMN, SCN and 
RDT&E Funds 

Prepare DOD Appropriations Act 
Prepare Appropriation Warrants 

Budget 28t* 

6.4.1 Call for Navy Education and Training 

Apportionment Request 290 

6.4.1.1 Modify Guidance from Budget Formulation 

to Fit Apportionment 292 
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6.4.2.5 Provide Information on Relmbursables from 
Appropriate Sources 
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6.6.3.6 Assess .r, -.effaces for Timing Relationships 

6.6.3.7 Assess Inter f*ces for Overall Workability 

6.6.3.8 Define Problems 1n System/Subsystem 
Relationships and Propose Solutions 

6.6.4 Evaluate System Inputs, Outputs, and Processes 

for Effectiveness 332 

6.6.4.1 Review Overall System Objectives 

6.6.4.2 Analyze Outputs of Established Monitoring 
Techniques 

6.6.4.3 Assess Outputs for Agreement with System 
Objectives 

6.6.4.4 Assess Outputs for Agreement with User Needs 

6.6.4.5 Assess Intra-System Interfaces 

6.6.4.6 Assess Interface to Overall System 

6.6.4.7 Define Problems within System and Propose 
Solutions 

6.6.5 Analyze Trends and Determine Future System 

Needs 334 

6.6.5.1 Maintain Awareness of Applicable Techniques 

6.6.5.2 Maintain Awareness of Trends 1n Data Pro- 
cessing Capabilities 

6.6.5.3 Maintain Awareness of Trends in User Needs 

6.6.5.4 Maintain Awareness of Problems/Studies and 
Solutions 
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6.6.5.5 Conduct Analyses and Studies to Define 
Future System Needs 

6.6.5.6 Develop Potential Solutions to Future 
System Needs 

6.6.6.7 Compare Trends, etc., to Proposed 
Solutions on a Continual Basis 

6.6.5.8 Formalize Solutions to Meet Identified 
RMS Needs 

6.6.6 Develop Resource Management System 

Improvements 336 

6.6.6.1 Evaluate and Classify Problems and 
Proposed Solutions 

6.6.6.2 Prioritize Overall Needs for Improvement 
to RMS 

6.6.6.3 Select Areas for Further Development 

6.6.6.4 Insure Development Will be Consistent 
with Objectives 

6.6.6.5 Develop Final Proposals to Justify 
Funding Request 

6.6.6.6 Seek Required Funding 

6.6.6.7 Establish Final Overall Development 
Schedule 

6.6.6.8 Perform System Analysis to Insure User 
Needs are Met 

6.6.6.9 Develop Specifications for System Improve- 
ments 

6.6.6.10 Develop RMS System Improvement 

6.6.7 Implement Changes to Resource Management 

System 338 

6.6.7.1 Insure Adequacy of User Personnel 

6.6.7.2 Train User Personnel on System Modification 

6.6.7.3 Verify Phase-In Schedule is Realistic 
(Modify as Required) 

6.6.7.4 Install Improvement - Parallel Operations 
1f Required 

6.6.7.5 Checkout and Validate Improvement Meets 
Specifications 

6.6.7.6 Correct Any Identified Deficiencies 

6.6.7.7 Turn Operational Responsibility Over to 
User 
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Functional Description of NETS - Function 6.0 Manage 
Training Resources 

A general description of this function was presented 1n the 
outline and top-level flow of the preceding section. Further 
detail 1s shown 1n this section through a series of flows 
which expand key subfunctlons. In addition to notes within 
the flows that elaborate on significant points, a general 
discussion of the flow 1s contained on the facing page. 
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Summary of Major Technological Gaps and Problem Areas - 
Function 6.0 Manage Training Resources 

A number of problems can be identified from sampling observa- 
tions throughout the Naval Education and Training System. 
The importance which should be attached to their correction 
is so Interrelated with, and dependent upon, changes In other 
functional areas that the order in which they are listed is 
only an approximation of importance (first being most 
important) . 

1) An effective system 1s needed for prioritizing training 
requirements so that in the continuing absence of the 
desired level of training funding, appropriate decisions 
can be made as to what will be funded within the Resource 
Management System (RMS). 

2) A reduction in the complexity of the RMS within the Navy 
is needed. The number of ways 1n which resources are 
categorized, I.e., by program element, function sub- 
portions, budget classification code, element of expense, 
etc., seems unduly complicated for the most effective 
application of the system at all levels. 

3) Resource management reports need to be made more timely, 
more accurate, and more relevant to the command level at 
which they are used. Performance reporting (actual vs 
plan) should have increased emphasis in terms of auto- 
mated and timely feedback to reduce the reliance on manual 
logs and analyses. 

4) The interface between the technical and resource subsystem 
needs strengthening to insure adequate definition of 
training resource needs during the planning and pro- 
gramming phases. 

5) A more well-defined accounting unit is needed for planning, 
tracking, and controlling training. Present units, such as 
average under instruction, average on-board, and student 
man-months, are subject to varying interpretations which 
preclude an accurate measurement of actual training. 

6) Greater and more uniform automatic data processing capability 
is required to adequately service all command levels. The 
nonuniformity and the absence of sufficient processing 
capability, however, is especially noticeable at the lower 
command levels. 
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7) Closely related to the preceding items 1s the need for 
faster access to resource data. This capability is 
especially important for detecting and analyzing 
variance, costing training plans, responding to queries, 
etc. An effective system for identifying marginal costs 
1s necessary for pricing plan alternatives and fluc- 
tuations. 

Idealized Approaches - Function 6.0 Manage Training Resources 

The objective of any resource management system should be to 
supply management tools which optimize operations in terms of 
planning and controlling resource usage. Improvements to the 
present system used within Navy training are proposed to meet 
this objective. One of the key assumptions which is relevant 
to the high emphasis on Improving the present system is that 
resources will continue to be highly constrained. Improving 
the overall cost-effectiveness of operations will be of 
primary importance to training management. In the context of 
the preceding, the following direction is proposed. 

1) Improve the planning system. A major objective of resource 
planning 1s to identify requirements early enough so that 
resources will be available when needed. This infers an 
emphasis on longer-range planning and a commitment of 
management to the process and its Implications. The 
process involves a system of early identification of re- 
sources. Effective communication paths and monitoring 
agents art> essential. The implications are that: 

a) Early planning must be accomplished 

b) Alternatives must be considered 

c) Realistic costing must be performed 

d) Justification must be improved. 

Early planning implies an earlier involvement in training 
plan definition. This has been covered in the discussion 
of Function 1.0 Develop Training Requirements and Function 
3.0 Analyze and Plan Training. 

Realistic costing means the availability of historical costs 
and planning factors as well as a system for modifying them 
to the needs of any new costing exercise. The alternatives 
are most effectively developed if technical planning personnel 
know not only the technological implications of their 
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decisions (plans), but also the resource Implications. 
The ability to better define alternative strategies and 
their related costs is closely associated to the ease by 
which a plan can be justified. 

2) Improve the resource prioritization process. It 1s reason- 
able to assume that the available resources will generally 
be less than the desired resources. This means that 
resource management continually involves an ordering of 
the desired activities Into some priority sequence. The 
present system of setting priorities does not appear to 
permit an effective match between available resources and 
tasks/activities/programs that will be accomplished with 
them. The present OPNAVINST 1500.33 - Navy Training 
Priority System - allows training courses to be ranked 
according to several parameters. Its application appears 
to provide an objective basis for elimination of courses* 
but this 1s questionable. A better measure would be In 
terms of some analysis of Fleet needs for specific types 

of trained individuals. 

Two factors important to the ultimate decision to either 
reduce or eliminate a training course are: 

a) The number of personnel presently performing the job 
as compared to the number of personnel actually re- 
quired. 

b) The general experience level of the personnel perform- 
ing the job. An assessment of this ijpe would require 
a much better communication path with the Fleets and 
with NAVPERS. This, however, not only benefits the 
prioritization process, but 1t assists the planning 
and control (establishment of realistic and manage- 
able quotas) processes as well. 

3) Improve the resource control system. An effective resource 
control system, among others, requires two things: (1) 

a sound understanding of the present rate and level of 
expenditure, and (2) the flexibility to redistribute re- 
sources to get the most out of those that are available. 
Neither of these appears to be accomplished in an optimal 
manner today. The reports intended to indicate rate of 
expenditure (NAVCOMPT 7000-9 Performance Statement* as an 
example) are not timely enough and are based on a linear 
(same amount each period), rather than on any monthly 
expenditure plan. As a result, substantial manual effort 
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is devoted to keeping the books. The reports used to 
accomplish this are basic accounting documents which 
should be used for analysis once a problem has been 
Identified through a higher level of reporting. 

Ideally, a sound plan should Identify the level and 
timing of resources as close as possible to what is 
expected. The actual usage of resources should bo 
matched to this plan and the variance identified as a 
result. Also, the availability of the reports should 
be such that time exists for corrective action to be 
taken, if necessary. 

It is assumed that the use of certain resources will 
continue to be tightly governed by law, therefore, the 
optimal degree of flexibility required for the most 
efficient management of resources may not be achievable. 
Even with this as a constraint, desirable rearrangements 
of resources should still be made. Flexibility is closely 
related to the ability to assign realistic priorities. 
Only then can a higher level command make an intelligent 
determination that resources should be moved from one area 
to another. It is recommended, however, that more timely 
and more meaningful reporting be instituted, especially at 
the lower echelons of command. 

4) General Areas 

a) Personnel. Ideally, all personnel involved in resource 
management would have a fairly high level of skill in 
financial planning and analysis. While a number of 
very competent individuals were observed in the per- 
formance of this activity, a wide variation was found. 

This professional activity should receive more emphasis 
in terms of the personnel selected and of the training 
they should receive in order to perform their jobs 
more effectively. 

b) Paperwork. It appears that a high percentage of the 
mnua) activity devoted to preparing basic documents 
could be more fruitfully applied in the analysis and 
solution of resource management problems. This, of 
course, implies either the elimination or the auto- 
mation of much of it. Some of both appears desirable. 

c) PPBS cycle. This is sometimes referred to as the 
"never ending 1 ' budget cycle. There appears to- be an 
imbalance between the amount of time devoted to pre- 
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paring programming and budget documents as compared 
to the time applied to analyzing resources usage. 
It would seem that the three resource input cycles 
(Programming* Estimating and Apportioning), represent 
an excessive amount of resource input effort. The 
consolidation of the estimate and apportionment 
cycles should be considered, at least at functional 
commands and below. 

d Changing requirements. An important aspect of resource 
management is the ability to rapidly assess the impact 
of changing requirements. Several things are implied 
in this capability: (1) marginal costing methods are 
available to determine incremental resource needs, 

(2) the major constraints are well identified, and 

(3) a system exists for performing the logic and 
computations rapidly. There are a number of paperwork 
systems available or under development which attack 
this problem to a degree. Ideally, however, more 
optimized solutions are attainable through sophisticated 
computer models which can treat the more complex inter- 
relationships among variables and the costs which vary 
as step functions. 

e Flexible budgeting. Akin to the marginal costing 
capability mentioned above, is the ability to maintain 
a flexible budget. It is assumed that there will be 
increased scrutiny of resources expended in comparison 
to actual output as opposed to the initial plan. A 
knowledge of the variable costs associated with any 
change in output (the primary output of training 
would be students, student days, or average on board) 
is necessary to adequately match actual output and re- 
source usage. Increasing congressional Interest in 
student output may raise this as an issue. At any rate, 
in an ideal situation, marginal costs should be readily 
determinable and available to planners and managers* 
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Function 7.0 Perform Training Research 

a. Definition 

Training research includes all activities concerned with 
the determination of objective facts/knowledge relating 
to the training/learning/evaluative process. Such 
activities are not confined to formalized experimental 
situations but include field studies, systematic analyses, 
statistical analyses (other than routine enumerations}, 
exploratory development, advanced development, and engi- 
neering development. Thus the concept of research is 
broadened to cover those activities designed to solve 
particular problems by means of objective methods, as 
well as to develop a base of knowledge and facts. 

The primary document for the research program in CNET is 
CNTINST 3910.1 (10 December 1971) - ''Training Research 
Program. " This establishes policy and describes the 
organization and requirements to meet research policy 
needs. 

b. Description 

The Perform Training Research function is described through 
an outline which contains all of the functional elements 
presented in the functional flows, and in addition through 
an overall functional flow, Figure VI 7.0 - Perform Training 
Research. 

Page 

7.0 Perform Training Research , . . . ♦ 347 

7.1 Develop Research Requirements 350 

7.1.1 Request Research Requirements 

7.1.2 Originate Research Requirements 

7.1.3 Coordinate Research Requirements 

7.1.4 Eliminate Obvious Duplicates 



N7.1A Develop Research Requirements (Idealized) . . 352 
N7.1.1A Survey Each Function/Subfunccion of NETS 
N7.1.2A Determine Significant Problem Areas 
N7.1.3A Survey Technical Literature 
N7.1.4A Survey Other Applications - Military, 
Industry, and Education 
J N7.1.5A Evaluate Implementation Requirements 
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Page 

7.2 Analyze Research Requirements 354 

7.2.1 Group into Related Problem Areas 

7.2.2 Develop Logical Relationships Within Groups 

7.2.3 Develop Priority of Problem Groups 

7.2.4 Select Highest Priority Problems 

7.3 Develop Research Plans 356 

7.3.1 Compare Priority Areas to Current GOR's, 
SOR's ADO's 

7.3.2 Indicate Needs to ONR 

7.3.3 Monitor Status of the Request 

7.3.4 Prepare Detailed Proposal Justification 

7.3.5 Request NAVOP 098 Budget 

7.3.6 Task Proper Research Agency 

7.3.7 Subcontract Research Project 

7.4 Perform Research 358 

7.4.1 Define Problem 

7.4.2 Develop Hypothesis and Design the Study 

7.4.3 Develop Study Approach 

7.4.4 Determine Required Resources 

7.4.5 Outline Analytic Approach-Detailed Plan 

7.4.6 Develop Schedule and Milestones 

7.4.7 Acquire Resources 

7.4.8 Initiate Pilot Study 

7.4.9 Revise Study Approach 

7.4.10 Initiate Data Gathering 

7.4.11 Complete Data Gathering 

7.4.12 Analyze Data 

7.4.13 Develop Initial Conclusions and Recommendations 

7.4.14 Write Study Report 
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N7.5.1A Determine if Results Achieve the Requirement 
N7.5.2A Determine 1f Current Method Achieves the 
Requirement 

N7.5.3A Determine Implementation Requirements 
N7.5.4A Compare Effectiveness - Current and New 
Methods 

N7.5.5A Determine Cost - Current Method 
N7.5.6A Determine Implementation Costs - Mew Method 
N7.5.7A Determine Costs/Effectiveness - Current and 
New Methods 

N7.5.8A Select the Most Effective Method Per Cost 
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Functional Description of NETS;- Function 7.0 Perform 
Training Research 

A general description of this function was presented 1n the 
outline and top-level flow of the preceding section. 
Further detail 1s shown 1n this section through a series of 
flows which expand key subfunctions. In addition to notes 
within the flows that elaborate on significant points, a 
general discussion of the flow 1s contained on the facing 
page. 
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Summary of Major Technological Gaps and Problem Areas - 
Function 7.0 Perform Training Research 

1) There 1s no apparent over-all research plan or approach 
1n the training system. What does appear 1s a series 
of Isolated and unrelated efforts. As a result, there 
1s duplication or great over-lap within the Navy and 
within services. 

2) There 1s no central research coordination of on-going 
studies for establishing resource priorities, systematic 
development of needs, or dissemination of study results/ 
Implications. 

3) The utilization of research/study results 1s very 
spotty. Actually, few such Instances were found. 
The norm 1s for the results to He fallow on a shelf, 
soon to be forgotten. 

Idealized Approaches - Function 7.0 Perform Training 
Research 

The rationale for Idealized approaches for training research 
stems from the major problems pointed out 1n the previous 
section (d.). The basic requirements are for systematic 
development of research needs, coordination and control of 
research efforts, and a method of Implementing worthy findings. 

1) Most of the research that 1s currently conducted seems 
to be 1n response to specific requests. Such requests 
emerge from various conferences of training commanders, 
the Ideas of a commanding officer or a staff officer, 
or the request of other Individuals in a position to be 
heard. The "needs" that emerge are often not well de- 
fined, may not be real needs, and may be complete 
duplicates of other stated needs. 

In other Instances, the research groups come up with 
their own Ideas of what should be worked on. The 
danger here Is that the potential users are not aware 
of the ongoing research efforts and/or don't agree that 
the "problem" 1s a significant one. 

Ideally, there must be an Initial systematic development 
of knowledge concerning problem areas and technological 
gaps. The present study 1s one aspect of such an In- 
vestigation, but since 1t concentrated on describing the 
Naval Education and Training System as a totality, such 
gaps were not the principal focus. A detailed delineation 
of problem areas and a specification of priority are 
essential for training research. The performing of 
all research within such a framework is essential 1f 
effective and useful results are to be attained. 
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Briefing of potential sponsors must cover, as a minimum, 
the goals of the study and the possible benefits. The 
sponsor should commit himself to the implementation of 
viable results. "Viable" means that the Implementation 
will yield increased cost-effectiveness as compared to 
the current approach. 

An Idealized approach for determining 1f a research out- 
put 1s viable 1s shown 1n Figure VI N7.5A. It 1s 
essential that the nature of the need 1s specified. If 
the current method (or the "new" method) does not achieve 
the objective 1t 1s supposed to perform, the method must 
be discarded, regardless of cost. If both methods 
are effective, a method of evaluating effectiveness 
must be devised. This may be done by rating, by 
establishing a percentage of effectiveness, or by any 
other appropriate method. Once the degree of effec- 
tiveness 1s established, the price of each must be 
developed. Then a comparison of cost per "unit of 
effectiveness" may be established. It 1s Important 
to understand that the cheapest methods may be more 
costly In the long run If the effectiveness 1s low. 
If the effectiveness is zero, even a very low price 
becones prohibitively expensive. 

Note that the functional flows do not specify which 
organizations perform functions or subfunctlons. 
However, in the instance of applying research findings, 
it would be desirable for the research organization to 
be directly Involved in the foregoing analysis. In 
addition, it 1s the best qualified to present the 
requirements for implementation as well as to perform 
the evaluation and initial support for implementation. 

While the kinds of problems and variables need system- 
atlzatlon as pointed out previously, the broad areas 
upon which i.o focus training research in the Idealized 
context are: 

Effectiveness Measures 
Task Analytic Approaches 
Feedback 

Performance Aids 

Training Device Application 

On-Board Training 

Individualized Instruction 

CAI/CMI 

Counsel ing 

Comments on most of these areas are to be found in the 
body of the discussions for specific functions, particu- 
larly Functions 4.0 and 5.0. 
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Function 8.0 Support Training 

a. Definition 

This function covers all activities directly relating to the 
support of training operations. In addition to supporting 
activities, there 1s also the development of: training 
packages for On-Board Training (OBT) and non-resident in- 
struction, as well as new training equipments. A significant 
subfunction is the procurement and maintenance of training 
devices, particularly complex simulators/trainers. 

While the support function is concentrated in the organization 
of the Chief of Naval Training Support, various subfunctions 
and tasks are conducted at all levels within the total train- 
ing system. In addition, all levels are participants in this 
function through generating requirements, as users, and as 
significant sources of feedback on the quality of training 
support. 

b. Description 

The Support Training function is described through an outline 
which contains all of the functional elements presented 1n 
the functional flows, and in addition through an overall 
functional flow, Figure VI 8.0 - Support Training. 



Page 

8.0 Support Training . 367 

8.1 Develop/Acquire/Support Training Material .... 370 

8.1.1 Receive and Analyze Requirements 

8.1.2 Determine Characteristics 

8.1.3 Develop Training Equipment 

8.1.4 Procure Training Equipment 

8.1.5 Maintain Training Equipment 



N8.1.1A Receive and Analyze Requirements (Idealized) . 372 
N8.1.1.1A Determine Specific Problem 
N8.1.1.2A Verify Need 
N8.1.1.3A Determine Potential Approaches 
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8.2 Support Training/Education Programs 374 

8.2.1 Support Non-Resident Instruction Program.. . . 376 

8.2.1.1 Develop Instruction Package 

8.2.1.2 Disseminate Study Materials 

8.2.1.3 Evaluate Student Responses 

8.2.1.4 Develop Student Record Data 

8.2.2 Support On-Board Training (OBT) 378 

8.2.2.1 Receive and Analyze Requirements for OBT 

8.2.2.2 Develop On-Board Training Packages 

8.2.2.3 Disseminate On-Board Training Packages 
N8.2.2.4A Evaluate OBT Packages 

8.2.3 Support Education Programs 380 

8.2.3.1 Develop/Support Policy for Education 
Programs 

8.2.3.2 Develop Education Programs 

8.2.3.3 Administer Education Programs 382 

8.2.3.3.1 Establish Prerequisites 

8.2.3.3.2 Develop Additional Resources 

8.2.3.3.3 Establish Quotas 

8.2.3.3.4 Administer Students 

8.2.4 Support Library Programs 384 

8.2.4.1 Determine Needs 

8.2.4.2 Procure Needed Books/Supplies 

8.2.4.3 Disseminate Books/Supplies 

8.3 Administer Advancement in Rate Testing Program . . 386 

8.3.1 Develop Examinations 388 

8.3.1.1 Analyze "Qualifications for Advance- 
ment in Rating" 

8.3.1.2 Write Test Outline 

8.3.1.3 Write Test Items 

8.3.1.4 Compile Test Battery 

8.3.1.5 Administer Test 

8.3.1.6 Analyze Items 

8.3.2 Develop Advancement in Rate Study 
Bibliography \ . . 390 

8.3.2.1 Compile Pertinent References 

8.3.2.2 Determine Applicable Sections of Each 
Reference 

8.3.2.3 Maintain Current References for Each 
Succeeding Examination 

8.3.3 Develop Test Statistical Data 392 

8.3.3.1 Determine Characteristics of Test 
Takers by Test Performance 

8.3.3.2 Perform Collusion Checks 

8.3.3.3 Perform Other Pertinent Analyses 
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8.4 Support Navy Training Publications 394 

8.4.1 Develop Rate Training Manuals (RTM) 396 

8.4.1.1 Evaluate "Qualifications for Advancement 
in Rating" 

8.4.1.2 Develop Manual Outline 

8.4.1.3 Write Manual by Chapters 

8.4.1.4 Review/Revise/Approve 

8.4.1.5 Publish and Disseminate RTM's 

8.4.2 Develop and Maintain Formal Schools Catalog . 398 

8.4.2.1 Obtain Inputs From "Schools" 

8.4.2.2 Organize Inputs 

8.4.2.3 Index and Cross -Reference 

8.4.2.4 Publish and Disseminate Formal Schools 
Catalog 

8.4.2.5 Collect Change Data On a Continuous Basis 

8.4.2.6 Develop Quarterly Change Publication 

8.4.3 Develop and Maintain Technical Publications 
for Training Devices 

8.4.4 Develop and Maintain Catalogs/Indices of 
Training Materials 
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Functional Description of NETS - Function 8.0 Support 
Training 

A general description of this function was presented in the 
outline and top-level flow of the preceding section. 
Further detail is shown in this section through a series 
of flows which expand key subfunctions. In addition to 
notes within the flows that elaborate on significant points, 
a discussion of the flow is contained on the facing page. 
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Summary of Major Technological Gaps and Problem Areas - 
Function 8.0 Support Training 

1) There 1s no systematic analysis for developing needs/ 
requirements for training equipment (aids/devices). 

r' 

2) The dissemination of Information about training equipment/ 
aids developed 1n individual schools Is limited. 

3) The development and support of on-board training requires 
a more Intensive emphasis. 

4) The school catalog program is cumbersome and difficult 
to use 1n Its present form. 

5) There are major discrepancies between naval policy on 
testing for advancement 1n rate and what the Individual 
actually does on the job. 

6) The Inventory and tracking of training equipment requires 
modernization. 



Idealized Approaches - Function 8.0 Support Training 

The approaches to Idealization are designed primarily toward 
the alleviation of major problem areas. However, several 
new points emerge which go across specific problems. Since 
these permeate the totality of the NETS, these non-functional 1 
specific areas will be Included 1n this section. 

1) A basic requirement for idealization 1s in the way 1n 
which needs for training equipment are developed. Train- 
ing equipment Includes various aids such as outlines, 
charts, slide-rules, nomograms, conversion tables or any 
other methodology which will help the Individual learn or 
perform on his job more effectively. This description, 
in effect, makes no distinction between training aids and 
performance aids. The assumption is that the training 1s 
aimed at job performance ultimately. Training equipment 
further Involves devices such as part task trainers and 
complex simulation devices. 

The more complex, primarily hardware developments, are 
discussed in Function 1.0 Develop Training Requirements, 
since such developments are handled, usually, as part of 
a total system procurement. However, the development of 
needs for other kinds of training equipment is through 
individual schools or other training organizations. These 
should be developed and analyzed in a systematic manner 
such as that shown in Figure N8.1.1A. Preceding such an 
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analysis, there should be a careful examination of a 1 l 
curricula and presentations to determine where aids may 
be helpful, Effective feedback from the Fleet should in- 
clude information on performance which presents unusual 
difficulties. Such information would indicate a potential 
need for an aid, in addition to a possible need for 
emphasis in training. The inter-relationships between 
instruction and aids is discussed in the context of Func- 
tion 4*0 (e) Implement Training. 

Several instances of excellent aids were found during 
visits to schools. In no case was it evident that informa- 
tion about such equipment/approaches/ideas had been commu- 
nicated to other training sites which might have use of 
such ideas. As an ideal, the total training establ ishment* 
as well as the Fleet, should develop needs for aids, as 
covered previously. Also, where so desired, each organiza- 
tion should develop their own Ideas, equipment, aids, etc. 
However, once these are developed, the information mast be 
made available to any and all segments that may conceivably 
use such an approach. 

There are several possible avenues of communication. 
One could be through a section of the NAVTRA magazine, 
which would give a general description of the approach, 
how it operates, and how to get more details on the 
idea. 

The catalog of training aids/devices should include in- 
formation about those self-developed by training units 
or other activities. Generally the production of aids/ 
devices should be through the Naval Training Equipment 
Center (NTEC). However, 1f the development of quantities 
is not justified -- based on a cost/utilization study — 
directions for "self-building 1 ' the item should be avail- 
able. Such "plans" would be listed. 

Periodic seminars and meetings, such as the Training 
Commanders Conferences should include presentations of 
training/performance aids. For example, several out- 
standing approaches to aids have been developed at NPRDC, 
It would be a virtual certainty that the specific aids and 
the approach would be utilized if the "school-house level" 
management had knowledge and understanding of these ap- 
plications. One way to build this comprehension would be 
through meetings which involve presentations on various 
techniques and methodologies, including the use of aids. 
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3) On-board training (OBT) is a direct Fleet responsibility. 
Since training is involved, this must be considered as 
part of the total NETS. The conduct of OBT is covered, 
indirectly, in Function 4.0 Implement Training. Aspects 
of OBT pertinent to the function of Support Training 
deal with the development of training courses, sections of 
courses, study guides, and references, as well as the aids 
mentioned previously. Such developments are needed to 
support OBT. 

The initial requirements for OBT must be developed system- 
atically. In no case should the Fleet simply be asked, 
"What do you need? 11 In one instance where such a request 
was made, a list of seventy-three items was returned. In- 
cluded were such things as: 

Cooking and Baking 
Basic Barbering 
Honors and Ceremony Officer 
Beach Gear. 

Other items further indicated that asking the Fleet was 
not the best way to define requirements. This is an 
example of the kind of problem mentioned in the Function 
1.0 discussion. 

Ideally, such needs should be developed on the basis 
of observed deficiencies in performance. Such a 
deficiency could be a failure to keep certain equipment 
operational within design parameters. Assuming the equip- 
ment does not have inherent design problems, it is logical 
to assume a training gap. The analysis implied here must 
clearly define and bound the need. The next question is a 
decision as to whether an existing course is applicable 
and feasible or if OBT is required. If OBT is needed, the 
basis for a clearly defined and specific training require- 
ment has already been developed. 

The development of OBT materials or training packages 
should generally be in conjunction with Fleet representa- 
tives or at least with close liaison. Regardless of how 
good the training support is, the Fleet must have some 
involvement. Otherwise, it is a distinct possibility that 
the materials may not be used. Materials for OBT must 
necessarily be of top quality since they will be largely 
stand-alones without the need for instructor intervention. 
The packages should be developed by professional per- 
sonnel , with assistance from technically qualified experts. 
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All such packages must be verified before release by a 
try-out in the form of an initial study. Such trial will 
show the need for revision, in all probability. 

As the use of OBT is stressed (see idealized approaches 
for Function 4.0), the needs and motivation for such 
packages will Increase A large percentage of the effort 
of training units in the future should be in the prepara- 
tion of OBT materials in support of the Fleet. 

The school catalog program in its current form has a large 
number of problems. This is a known fact in the Navy, 
both within and outside of the NETS. A basic requirement 
is standardization of: 

Course numbering 
Descriptive information 
Indexing. 

Such standards, among other things will eliminate a fair 
amount of duplication which exists. Standards and a care- 
ful definition of terminology will be an early step 1n 
ultimate computerization of all training catalogs, Including 
those for technical publications, films, training equipment, 
etc. 

The development of a computer based system for cataloging 
is an essential idealization. It would make the informa- 
tion easy to obtain, would facilitate revision, and 
eliminate much confusion. 

This system, while having facility for physical printout 
of volumes, should also be capable of query by remote 
terminals. If an Individual wishes to learn what courses 
are available for a particular kind of job in relationship 
to a specific system, he should be able to get just that 
Information, including when the courses meet, who has 
quota control, and how to apply. 

The dichotomy between job performance and the qualifica- 
tions for advancement must end in the idealized situation. 
Advancement in rating exams could be based upon the PQS/ 
Task Analysis. The most significant advance to be made is 
to require that the prerequisite to testing be a demonstra- 
tion of proficiency upon selected parts of the pertinent 
PQS. 

While several possible approaches might be used, one po- 
tential alternative would be to either use a single task 
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analytic approach for both training and ;he"Qual if Ra- 
tions for Advancement in Rating," or to remove the use 
of the "Quals" as the basis for testing. 

The use of task analytic data may require a more complex 
kind of test situation, since both general and equipment 
specific kinds of knowledge may have to be tested. For 
example, fifty percent of a test might cover items applic- 
able to the entire rating,, while the other fifty percent 
might pertain to individual equipments or groups of equip- 
ment. Some similarity car be obtained by using an outline 
which covers the same general kinds of information about 
different hardware. With the use of standard scores and 
statistical corrections, if needed, a more individualized 
testing procedure is certainly feasible. 

Gearing the testing program to ,;ob performance would provide 
a strong positive incentive for OBT of all types, but 
particularly for the PQS approach. If a change in testing 
procedures and policy is not made, the total test program 
may be regarded increasingly as a blind obstacle, with a 
merely formal goal of making promotion an added hurdle un- 
related to performance. Since the Navy program is the 
best among the services, all effort should be made to keep 
that image. In part, there is a recruiting incentive in a 
fair, intelligent method of selecting personnel for advance- 
ment. 

Another idealized approach to the examination program would 
be greater objectivity in selecting personnel that are 
recommended to take the examinations. This may be achieved 
by requiring, as a prerequisite, the completion of pertinent 
parts of the PQS. 

The inventory of training equipment of all types is ex- 
tremely large and varied. Revision of records is neces- 
sarily complex and lengthy under such circumstances, In 
addition, the problem of keeping track of the large variety 
and numbers of items which have been assigned or loaned is 
truly gigantic. The ideal approach consists of computeriza- 
tion of this mass of data. Once the techniques are set, 
there would be no need for large volumes of paper records, 
since flexible retrieval would allow the presentation of 
only that information which is needed. In addition, pro- 
grams can be instituted which would provide automatic 
printouts of data on a specific time basis. This kind of 
application amounts to an automatic suspense file, for 
determining, on a periodic basis, where certain items are, 
or presenting the location of items which require periodic 
preventive maintenance. 
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01SCUSSION 

This report presents a generalized functional description of the 
Naval Education and Training System which has been designated as 
the NETS. The major segment of the NETS is the Naval Education 
and Training Command (NETC). Consequently, the focus was on that 
command. However, a significant amount of training takes place out- 
•side of that command and much of this other training was included in 
this study. 

Not all parts of the NETS were visited. The data which form the 
groundwork of this report were obtained on a representative sampling 
basis. Specific organizations visited are shown 1n Appendix 0. 
While a large number of organizations were visited* 1t is clear that 
many important areas were not. There is no significance in the fact 
that an organization was or was not visited, The objective was to 
get a representative cross-section of naval education and training, 
and at the same time have an analytic task that would be feasible. 
The cross-section was obtained by visiting at least one of every 
type of training organization. Feasibility involved the arranging 
of travel and duration of stay at a location In the most cost-effective 
manner possible. 

The ultimate objective of this research project is to develop computer 
models as management tools with which to improve training. Phase I 
forms the foundation by describing the current system. This system is 
described functionally through a series of flow charts. The system 
was considered to have eight major functions. Each function is broken 
down systematically and then further analyzed to identify subfunctions 
and tasks. No consideration is given as to where, organizationally, 
these operations are performed. 

This point is significant, since another part of the Phase I study is 
to Identify technological gaps and problem areas. This identification 
is done by function. In no case do these gaps/problems pertain to any 
particular organization. They pertain, however, to the over-all 
system. To an extent some of the problems may relate to some or all 
organizations. However, it would be erroneous to assume that any unit 
or units were singled out. Because of the need to cji ! i;*ralize so 
widely, it was necessary to be all-encompassing and to stress common- 
alty rather than nuances. The fact that an organize on does not have 
any one problem area mentioned doesn't mean that tht problem is not a 
system problem. It may mean that for that particular point, the 
organization has a system for effectively avoiding the problem. In 
this case, careful analysis of the approach used by that organization 
may help resolve the over-all system problem. 

A number of idealized concepts are given also for each function, as 
well as for critical interfaces with functions outside of the train- 
ing/education area. Since all significant functions are beino per- 
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formed* it was not necessary to add functions. Idealization centered 
around the methodology of performance and different techniques, 
particularly in an altered context to be expected in the 1980 time 
frame. Instances may be found where some approaches of various 
organizations come close to a few of the idealizations. This is 
to be expected and is a direct manifestation of differences between 
organizations. In no case can it be stated that any organization 
is near the ideal since most of the concepts are oriented in a 
future, rather than a current situation. 

As previously stated, the objective of the Phase I study was to set 
the foundation for Phase II - the development of mathematical models. 
However, there are direct current applications of the output of this 
report. Such applications may be categorized as: 

o Operational/Systems Analysis 

o Documentation 

o Communication 

o Training of Staff. 

1) Operational /Systems Analysis 

The analysis described in this report is a study of the operations 
of the Naval Education and Training System (NETS). The analytic 
approach was systematic in that a progressive functional breakdown 
was followed. Since this analysis covered a broad cross-section, 
many parts will not fit exactly for specific organizations. How- 
ever, by using this approach as a guide, individually tailored 
functional/operational flows may ba developed. These could follow 
a functional orientation and/or be based on specific operations. 
The most desirable technique is to follow a combination of these. 
The functional analysis assures that all significant functions/ 
subfunctions/tasks have been considered. In those instances where 
operations involve combinations of functions, flows for the opera- 
tions are required. 

All applications of this analytic approach require that such a 
"tailored" analysis be performed initially. A primary immediate 
advantage of this analysis is the fact that the organization can 
determine readily the effectiveness of its operations. Such evalua- 
tion is performed by looking for such problems as: 

o Redundancies 

o Operations/functions with no output or perceived purpose 
o Lack of inputs or appropriate/timely inputs 
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o Gaps or communication failures 

o Required functions not being performed. 

This is only a partial listing of problem areas which may be 
revealed by such a detailed analysis focusing on a single 
organization. The graphic form of the flows as carried to 
lower levels of detail magnifies any "system" operational 
difficulties which might exist. The initial application then 
is to allow a careful self-evaluation of the operation of an 
organization. 

Documentation 

A second type of application is as a form of highly detailed 
documentation. Organizations are invariably required to develop 
operating procedures. These are usually unclear, incomplete, 
difficult to understand, or obsolete. The system flows for an 
individual organization may be documentation which suffers none 
of these deficiencies. At higher levels of detail, each block 
tells what must be (or is) done. As the levels become increas- 
ingly more detailed, they eventuate into how to perform each task. 
Thus, carried far enough, procedures may be the final output. 
Documentation in the form of flows is relatively simple to revise 
and update, which is another advantage of such documentation. 

Communication 

The system flows are also one of the clearest forms of communica- 
tion that might be used. No other method can so lucidly convey 
alternative approaches, logical branching, and loop-backs. One 
page with a small number of blocks can convey a message that 
pages of text could not equal. For this reason, the flows are 
useful to describe new or revised approaches * as well as to plan 
potential approaches. Often a concept that initially appears 
sound will be revealed as flawed when placed in flow form. 

Training of Staff 

The preceding applications all point to the fact that appropriate 
documentation in the form of flows serves as an outstanding 
vehicle to orient/train new staff personnel. 

A new commanding officer may obtain an overview of his organization, 
as well as get a. good general feel for what is being done and how 
it is done* Similarly, new personnel may learn in a relatively 
simple manner how their duties fit into the total operation. 
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A number of potential applications have been mentioned. It 1s 
necessary to point out again that such application^ require an 
Initial detailed analysis of the organization. Such analysis 1s 
not a simple operation, nor does 1t come easily. Generally, trained 
professional analysts are required. There is a possibility that 
selected staff personnel, with training, may be able to perform 
some analysis with proper assistance and monitoring. Personnel 
selected must be highly proficient in the technical area to be 
analyzed; have an analytic, methodical personality; and be 
motivated to perform such analysis. 
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SELECTION AND EV ALUATION OF CANDIDATE MODELS 

A. INTRODUCTION 

A systems approach requires that suggested design alternatives be 
evaluated by formal prediction of possible consequences. The tools 
usually used for such predictions are called models. These may be 
mathematical constructs, computer simulations, graphical procedures 
or other techniques which aid in predicting the way in which any 
proposed system or subsystem operates. In addition to evaluating 
system design alternatives, predictive models serve an equally 
useful purpose in the management planning process. A management 
model may assist a school manager in evaluating the alternatives 
o.jen to him as a result of course cancellations or a requirement 
for additional offerings, or a model may be used to evaluate the 
feasibility of implementing a specific training plan throughout the 
training organization. Although predictive models classified as 
management planning models can take many forms, the objective of 
each is to provide the manager with the necessary tools to operate 
the existing system as efficiently as possible. Candidate models 
for future development of both types are described and evaluated in 
this section. 

B. GENERAL APPROACH 

This section describes the process by which a subset of the total 
potential modeling areas identified within the Naval Education and 
Training System (NETS) (described in Section VI) h*s been selected 
for development as a set of mathematical models. 

Figures VII 1 and VII 2 depict the top level functional elements that 
comprise the NETS. Both existing models and proposed candidate models 
are listed under the appropriate functional block. By putting each of 
the candidate models in perspective against the total training system, 
the NETS requirements in terms of needs and the interactions of 
candidate models > both within a functional area and between functional 
areas, can be better assessed. In addition, if it is assumed that all 
proposed candidates will eventually be developed and implemented, then 
Figures VI i 1 and VII 2 can be thought of as a composite model of the 
total training system able to predict the behavior of the system under 
a variety of conditions. 

However, given the constraints of time, money, and technology, only a 
subset of all potential modeling areas identified within the training 
system can be addressed. Since that subset will of necessity consist 
of only one or two of the modeling areas, both the process of identify 
ing the total set of modeling areas from which the final selection 
will be made, and the process of evaluating the modeling area to be 
developed during Phase II of the Design of Training Systems project 
are of prime importance. 
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As stated in the Phase II Management Support Plan* all candidates 
meet the following general criteria: 

1. The projected models should be feasible so that an early 
failure does not preclude continued efforts to improve the 
training system. 

2. The models should address existing problems so their practi- 
cality can be demonstrated* 

3. The models should be oriented toward problems which are unique 
to the training system so as to avoid duplicating models 
already in use for personnel or manpower management. 

The goal of the selection task then is to determine on the basis of 
current information which of the candidates best meet the above 
stated criteria. 

The candidate list presented in Section VII C represents the result of 
surveying many potential model users at the functional command level 
{within CNET) as well as several activities. Application of the 
survey interview technique provided the study team with a cross- 
section of both immediate and long-term problems experienced by each 
command. The candidate list was prepared based on these data together 
with an analysis of the functional flows described 1n Section VI. 

More specific criteria were generated based on the general criteria 
noted above. Because the candidate modeling areas have not been 
developed in sufficient detail to allow evaluation in terms of manpower, 
dollars, and effort, the candidates were subjectively evaluated in terms 
of risk factors and potential benefit based on the criteria described in 
Section VII E. 

C. CANDIDATE MODELING AREA DESCRIPTIONS 

The following list of candidate modeling areas and their respective 
descriptions were reviewed and approved by both the Navy Project 
Office and the Navy Working Group. No additions or deletions were 
made to the original candidate list which was submitted in the 
Phase I Preliminary Report. 

1. Training System Capabilities/Requirements 

A model to analyze the training requirements in terms of system 
capability would be based on a detailed training system inventory 
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1 n terms of facilities* equipment, and materials organized by 
course/type of instruction as well as an analysis which determines 
the requirements for facilities, etc., as a function of student 
enrollment on a course basis. The objective of this model is to 
assess the feasibility of meeting annual training requirements 
in terms of desired school throughput as well as to evaluate 
alternative implementation plans and their effect on the total 
training organization. Data base requirements for this model 
type would be met to a large extent by NITRAS (Navy Integrated 
Training and Resources Administration System) and CENTRA (Cen- 
tralized Training Material Management Subsystem) both of 
which are currently under development. 

2. Training Resource Allocation 

This model type is closely related to the Training System Capa- 
bilities model discussed above, but will be designed to operate on 
a more detailed basis at a lower organizational level. The 
model will primarily address instructor billet utilization at 
the school level as a function of training requirements. Various 
strategies to modify the existing operational training plan will 
be evaluated by the model in an attempt to determine the optimum 
strategy. As with the Training System Capabilities model, some 
portion of the data requirements may be fulfilled by NITRAS and 
CENTRA. 

3. Management of Congressional AOB Training Ceilings 

The Congressional AOB Training Ceiling Management model will inter- 
face with NITRAS to maintain current AOB counts by training 
category and organizational level. These data will be used to pro- 
ject potential problem areas with sufficient lead time to implement 
solution strategies. The model can be designed to test the effect 
of implementing various training strategies. 

4. Budget Analysis Model 

The objective of this model would be to compare the base data used 
to compute budget projections with actual data for the purpose of 
identifying major areas contributing to the overall deviation of 
the actual from the projected budget. This model would be imple- 
mented at the ONET level to explain past budget deviations as 
well as to defend budget projections. 

5. School Planning Model 

Because of changes in personnel attrition, operational commitments 
and overall personnel management policy, Fleet requirements for 
skilled personnel can often change unpredictably. 
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Since a large portion of "C" school entrants come directly from 
ll A" School, "C M school planning must be coordinated with "A" 
school planning as well as with Fleet requirements, "C" school 
planning is presently handled by BUPERS using a manual process. 
The proposed model would attempt to determine the relationship 
between all independent variables influencing all "C" school 
requirements. A potential outgrowth of this modeling endeavor 
is the modification of current "A u school planning techniques 
to incorporate "C" school requirements. The "C" school planning 
model would interface with portions of the Advancement, Strength 
and Training Plans (AOSTAP) System as well as with one or more 
modules of NAVTIS (Naval Training Information System). 

6. Aviation Training Management/Planning System 

The objective of this effort would be to utilize the data con- 
tained in NATIS (Naval Aviation Training Information Systems), 
which is currently being developed as part of NAVTIS, to con- 
struct a system which would dynamically schedule all aviation 
training. The Management/Scheduling System would consider 
variables associated with students, instructors, and equipment, 
including simulators and aircraft. A planning system based on 
NATIS and the management system would also be developed. The 
planning model would project the effect of quantitative changes 
in variables on training throughput rate. The model could also 
be developed to project required modification to the existing 
training system to meet a specified Pilot Training Rate (PTR). 

7. Optimal School Location Model 

A model would be developed to evaluate the effectiveness, in 
terms of training and cost, of consolidating or expanding training 
locations. Locations of school consolidations could then be 
optimally determined in terms of major parameters such as travel 
distance. An analysis resulting in the identification and defi- 
nition of optimization parameters would be performed in con- 
junction with the modeling effort. 

8. Educational Technology Evaluation Model 

This system of models would be used to perform educational 
technology trade-off analyses at the school level. Variables 
which determine the cost of training based on the utilization of 
various educational strategies (classroom lockstep, instructor 
managed instruction, computer managed instruction, computer 
assisted instruction) would be identified and used to compute 
factors such as the number of students for which the costs 
associated with two alternate strategies are equal to the savings 
per additional student over the breakeven number, cost per 
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student hour per strategy, etc. Additional consideration would 
be given to specification of equipment requirements for com- 
puter based systems. Also included within this system would be 
a model to evaluate the effectiveness of centralized versus de- 
centralized computer installation for computer based media. 

9. Course Costing Model 

Although model definition and data collection may represent 6 
major portion of the total modeling effort, the Course Costing 
model would have application in several areas. The model could 
be used to determine the relationship between student/course load 
and the total operating budget at various organizational levels. 
An optimal student/course loading could be established as a 
function of budget or physical (facilities, equipment) constraints. 
The Course Costing model could also be used in conjunction with the 
Educational Technology Evaluation model. The Course Costing model 
would also interact with the Training System Capability model in 
that suggested optimal loading may require a modification of the 
existing system configuration which in turn may impact system 
capacities. 

10. Fleet Training Requirements Projection Model 

Considering all types of naval training, from preparatory to 
basic ("A" school ), advanced ("B" school), specialized ("C" 
school), and Fleet training, and the many sources from which 
trainees are derived, Fleet input to specialized and Fleet train- 
ing courses is perhaps the most difficult to project - even on a 
short-term basis. This is not surprising, considering the large 
number of variables involved. Among the more important variables 
which need to be considered arc the Fleet deployment schedules, 
individual ship manning documents, level of ship's manning, ship's 
equipment, applicable training courses including schedules and 
locations, the level of training achieved by current ship 
personnel, rotation schedules, etc. The Fleet Training Require- 
ments Projection model would significantly improve planning capa- 
bilities at the school level, since projection of Fleet input 
represents a major deficiency in total school planning. 

11. Quota Generation Model 

Identification of training requirements can be viewed from two 
different points of view/ The Fleet Training Requirements Pro- 
jection mode! would bo utilized to protect a portion of total 
school ijiput currently labeled "unprogrammed training' 1 requirements. 
Training requirements can also be viewed as a requirement to pro- 
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duce a certain number of graduates of a certain type during a 
given time. Or, in other words, projection of required school 
output. Requirements for the latter case can be further sub- 
divided Into two areas: (1) training to meet new system/equipment 
requirements; and (2) force maintenance training requirements. As 
a result of Training Plan Conferences, comparisons with training 
requirements for similar equipments, ship's manning documents, 
etc., a new weapons system will generate a quota in terms of 
required NEC's which the training system must meet. A Quota 
Generation model would use these data in conjunction with the 
operational delivery schedule for the weapons system to provide 
a time based projection of required output from the training 
system. 

12. Career Path Model 

The career path of the naval enlisted man is determined by a 
number of decision points such as promotions, training received, 
and assigned billets. The criteria governing these decision 
points are often unrelated and even where they are related they 
may be applied by different organizations and at different times. 

For example: whether a man undergoes a particular type of train- 
ing may be determined by his assignment to a new billet and the 
accompanying Permanent Change of Station (PCS). However, assign- 
ment to a particular billet Is a function of his rate, but promo- 
tions (and hence rate) are determined by another set of criteria 
applied independently. Consequently, attempting to analyze or 
predict the results of a change in career structure requires 
study of more than one functional entity. Career structure 
changes can be studied either by replicating the system as it 
exists or by assuming an integrated system. 

A decision-tree type model depicting movement of individuals 
through the system could be constructed to evaluate the merit of 
present or proposed career structures. The model could be 
designed to reflect current assignment and promotion policies or 
to evaluate the system impact of modifying any of the current 
policies. Career path definition could represent a major portion 
of the total model development effort, 

13. Training Process Flow Models 

The title attached to this type of model provides an accurate 
literal description of its objective, the study of the flow of 
trainees through the naval training process. The design of a 
specific process flow model, however, must be oriented toward 
the solution of one or more problems which may either exist in a 
current system or are to be avoided in the design of a new system. 
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For example, a process flow node! of a specific school could 
be used to analyze the relationship between the school through- 
put rate, course structuring, equipment quantities and utiliza- 
tion, student arrival rates, etc. Once the relations between 
these variables are known it would be possible to design the 
training process to produce the desired annual school output 
while minimizing required equipment quantities and maximizing 
utilization, as well as meeting other specified constraints. 

Another process flow candidate model could be termed a gener- 
alized process flow model. Its objectives are similar to those 
of the specific school model discussed above, but the model flow 
is generalized to the extent that detailed design analysis could 
be performed for many schools. The model would be designed such 
that individual schools are described in terms of student inputs, 
course structures, equipment requirements, and other descriptive 
parameters, which comprise the total data input to the gener- 
alized flow model . 

As an alternative to modeling the flow of individuals through a 
particular school, an aggregate flow model would be used to study 
the flow of groups of trainees through several schools, from 
recruit training through rating achievement. Although not as 
specific as the entity flow model, many of the same type of 
questions can be answered through the implementation of the 
aggregate flow model . 

14. Management of Fixed Resources Model 

Although related to the Training System Capabilities model, the 
•objective of the Fixed Resource model is to analyze and manage 
the fixed resources such as facilities and equipment, required to 
meet training needs during periods of increasing and decreasing 
training requirements. The data base requirements for this model 
would be nearly identical to the Training System Capabilities model. 
The model would measure the impact of contracting or expanding 
fixed training resources upon the total training system. 

15. Schoolhouse Training versus Off-Site Training 

Off-site training refers to training in a location other than the 
traditional schoolhouse (i.e., shore -based or shipboard) , The 
training concept would be the same as computer managed instruction, 
except that the students would not congregate at a central loca- 
tion. Instruction utilizing a type of self-paced instruction 
would be controlled via a telecommunication terminal linked to a 
centrally located computer complex. The objective of the candidate 
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is to determine the hardware required to Implement off -site 

training and then determine the merit of the technique relative 

to schoolhouse training in terms of both cost and the effective- 
ness of training received. 

16. Minimum Level of Training 

It is felt that some of the training administered by the Navy 
could be reduced or eliminated while still maintaining a skill 
level sufficient to meet the Navy's operational needs. An 
analysis to determine the minimum amount of training required 
for Navy personnel for each rate and rating should be performed. 
The results of this study could then be transferred to the Train- 
ing Process Flow and Training Resource Allocation models to 
determine an optimum implementation plan for the indicated train- 
ing system modifications. 

17. Training/Billet Assignment System for Students Completing 
Individualized Instruction Curriculum 

With the advent of individualized/self-paced courses of Instruc- 
tion* graduation dates can no longer be established with absolute 
certainty. Unless a student is reassigned as soon as possible 
after graduation, many of the benefits gained through individu- 
alized instruction will be lost. A student who has completed the 
prescribed course of instruction but has not been reassigned may, 
while he remains idle between assignments, deprive another 
potential student of a slot in that school and/or the billet to 
which he will eventually be assigned will remain unfilled. The 
objective of this analysis would be to design and implement art 
assignment system which is compatible with self-paced training 
techniques. 

18. Permanent Change of Station (PCS) /Temporary Additional Duty (TAD) 
Budget Impact on Training 

Since most training assignments require either PCS or TAD funds 
to get the student to the training site, an analysis should be per- 
formed to determine what effect PCS and TAD fund limitation has on 
the overall training effort. The results of this analysis would 
represent a major segment of the analysis required to construct 
the Optimal School Location model. The initial task to be per- 
formed in the location model is to identify the variables to be 
optimized by the selection of a school site. 

19. Training Objective Analysis 

The major question to be addressed by this analysis is: should 
training be performed solely to achieve Fleet performance or 
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also as an incentive for enlistment or reenl istment? The study 
would also cover such areas as the effectiveness of training as 
a reenl Istment incentive, comparison of training specifically 
performed as an enticement to reenl istment with training to in- 
crease Fleet performance, and costs of such training. 

20. NEC Manpower Overage 

The objective of this analysis is to design and implement a 
technique for handling manpower overages within an NEC. Among the 
options to be assessed in terms of both cost and training effec- 
tiveness are removal of manpower overage and retraining for a 
related NEC or, forget about NEC overage and underutilize personnel. 

21. Physics of Learning 

The Learning Process model would attempt to determine the relation- 
ship between training media and the learning rate of the student. 
The major obstacle to be overcome in the development of this 
model is the paucity of data required to establish the relation- 
ships. 

CANDIDATE SELECTION CRITERIA 

Three general criteria for candidate model selection were specified in 
Section VII B. These three general criteria formed the basis for the 
11st of specific criteria against which each of the candidates was 
evaluated. The first general criterion states that: 

1. The projected models should be feasible so that an early failure 
does not preclude continued efforts to improve the training 
system. 

Four of the selection criteria have been designed to insure that the 
selected candidate represents a feasible modeling task. These four 
include: 

Model development effort 

Model application/verification effort 

Technique availability 

Data availability 

As explained in the individual criterion descriptions, a higher rank- 
ing in these four criteria indicates a higher degree of feasibility. 



VIM1 



TAEG REPORT NO. 12-1 



The second general criterion states that: 

2. The models should address existing problems so their practi- 
cality can be demonstrated. 

Because of the approach used to identify the candidate models {i.e., 
the functional level survey interview), all candidates represent 
existing problems and hence their benefit to the training organiza- 
tion can be demonstrated immediately upcn implementation. Also an 
estimate of the relative potential benefit to be derived from each 
model application was made as a part of the candidate evaluation. 

And finally the third general criterion in Section B states: 

3. The models should be oriented toward problems which are unique 
to the training system so as to avoid duplicating models already 
in use for personnel or manpower management. 

The criteria entitled Project Definition is aimed at identifying 
functional responsibility for each of the candidate areas. All 
candidates are functionally oriented toward training; however, train- 
ing functions related to personnel or manpower management are per- 
formed within BUPERS. A high rating under this criterion indicates a 
problem area unique to the training system as well as one which has 
not been actively pursued by the training organization, 

E. CRITERIA DESCRIPTIONS 

1. Design of Training System Project Definition 



Since some training system functions, i,e.» training requirements 
generation and school assignment, are performed under the Chief 
of Naval Personnel* the objective of this criterion is to establish 
whether the candidate model development falls under the juris- 
diction of CHNAVPERS or CNET. Also considered in conjunction 
with the project definition criterion is the current state of 
development of the candidate area. Several of the candidate 
models are under various stages of development within current Navy 
projects. The candidates are classified on a continuous scale on 
the basis of the following completion states: 

HIGH Mew effort 



Navy project 



conceptual stage 



Navy project 



data base development stages 



LOW 



Navy project 



model development stage 
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Thus a candidate will receive a high rating of 9 if it represents 
a new project completely within the domain of CNET. 

2. Model Application Time Span 

An attempt has been made to assess the time span over which the 
candidate may remain operational without excessive modification. 
Also considered in the time span rating is the expected utiliza- 
tion frequency of the candidate model. The objective of this 
criterion is to differentiate between those models designed 
primarily to perform a one-time evaluation or trade-off analysis 
{rating * 1) versus those models which could be utilized on a 
nearly continuous basis through 1980 (rating = 9). 

3. Benefit Potential 

A major consideration in the identification of a modeling applica- 
tion should be the savings which could be realized through the 
development and implementation of the selected model. Since it 
is difficult at this point to identify specific savings in terms 
of dollars, all candidates have been rated on a scale of 1 - 9 
as to their potential benefit to the Navy training organization. 
Several variables were considered in arriving at the final rating. 
As a yardstick to measure relative potential of each candidate, an 
attempt was made, when possible, to estimate the percent of the 
training population addressed by the candidate. Also, an estimate 
of the percent of the total training budget addressed by each 
candidate was considered in conjunction with the population 
figure, since the cost of equipment and facilities varies greatly 
for the various types of training. Finally an attempt was made 
to estimate the training organizations' relative requirement for 
each of the candidate models based on the descriptive functional 
model described in Section VI and observations of the study team, 
A rating of 9 represents both a high savings potential and a 
high requirement estimate. 

4. Organizational Level 

The organizational level, i.e., CNET, functional, or activity, 
which will benefit from the development of each candidate model was 
taken into consideration in rating this criterion* In some cases 
a single model or simple modifications of that model may benefit 
several levels of training management. A high correlation should 
exist between organizational level and savings potential, in that 
decisions made at higher management levels will generally .impact 
a larger portion of the total training budget or of the total train- 
ing population. Thus models designed to assist in management 
decisions at several organizational levels received the highest 
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rating. Models to be Implemented at the CNET level received 
the next highest rating, while models to be Implemented at 
the lower echelon level received the lowest rating. 

5. Technique Availability 

Availability of a modeling technique is a highly significant 
criterion. If a technique does not exist to design and develop 
a candidate model (rating ~ 1) that candidate should be removed 
from consideration for near term development. Given that a 
technique does exist for model development, then the rating 
reflects an assessment of the risk Involved in successfully 
utilizing the unmodified technique. 

6. Data Base Availability 

The data base criterion can be subdivided into four categories: 
data available-computer source; data available-flat paper source; 
data base currently under development, e.g., Integrated Flight 
Training Information System ( I FT I S ) , Navy Integrated Training and 
Resources Administration System (NITRAS) ; and data not available. 
The rating was assessed on a continuous scale of 1 to 9 with a 
totally computerized data source available = 9 and nonexistent 
data =1. A candidate model could be excluded from further con- 
sideration only if the required data base does not exist. Even 
then, if the candidate was ranked relatively high on the basis 
of the remaining criteria, development of a data base may be 
warranted in conjunction with model definition and development or 
the data base development could be pursued as a separate project. 

7. Projected Model Development Effort 

Unless the projected development effort is much larger than that 
allowed by the project time frame, this criterion cannot 
eliminate a candidate model from development consideration. If 
the candidate is favorably assessed on the basis of the preceding 
criteria, the value of this criterion lies in the control of the 
scope of development. Rather than eliminate the candidate from 
consideration, a more prudent decision would be to modify the 
scope to match the available time. This criterion was rated on 
a scale of 1 to 9 representing a risk assessment of the potential 
for completing the development effort (without seriously modifying 
initial scope] within the project time frame. A rating of 9 
indicates no risk; a rating of 1 indicates a high risk. 

8. Projected Model Application/Verification Effort 

The effort associated with applying the developed model to a real- 
world problem, including any effort associated with maintaining 
or compiling the required data base, and verifying the results of 
the application has been separated from the model development 
effort. The purpose of rating these efforts separately is that, 
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in several cases, development effort is projected to be relatively 
low risk while application and verification effort could be a 
high risk effort. The reverse could also be true, that is, the 
model development is expected to be a complex task and* therefore, 
high risk, while the application of the node!, once it is 
developed, is anticipated to be straightforward with no complica- 
tions foreseen. Application/verification effort will be rated 
for each candidate based on the same scale used to assess the 
development effort. 



9. Model Objective 

For the purpose of this evaluation, model objective is divided 
into two areas: training system development or those models 
whose major objective is to evaluate alternate courses of train- 
ing system development; and management systems whose goal is to 
assist managers in the decision making process associated with 
the operation and planning of the training system. These two 
objectives are not necessarily mutually exclusive. A candidate 
may achieve both objectives or either one individually. No 
attempt will be made to associate a rating with individual 
objective achievement at this time; however, a candidate achiev- 
ing both objectives will be rated as more desirable than one 
meeting only one of the two objectives. 



CANDIDATE EVALUATION 

Using the candidate and criteria descriptions, all candidates were 
evaluated by each member of the Phase II study team. After the 
individual evaluations were complete, the results were discussed, 
modifications made, and the ratings were compiled to form the composite 
study team candidate evaluation shown in Table VII 1, 

Since the range of total ratings for the seven top ranked candidates 
was very small, it was decided to remove the fourteen lowest ranked 
candidates from modeling consideration within the Design of Training 
Systems project. Following the initial ranking of the candidate 
models, both the selection criteria and the remaining candidate models 
were discussed. Tho top seven candidates were then reevaluated to 
determine the final rankings. As a result of the revaluation the 
final rankings of the top seven candidates differ from the initial 
rankings of these same models. The results of the revaluation are 
shown in Tables VII 2 through VII 8. 



A summary of the major reasons for rejecting the fourteen Truest 
ranked candidates is presented. 
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Candidate Rejected Because 

M C" School Planning "C" school planning 1s currently a 

BUPERS function and, therefore, outside 
the realm of CNET. Also, the problem is 
b6ing investigated by researchers at 
NPRDC in San Diego. 

Optimal School Location It was assumed that the location model 

would be used very infrequently. There 
is also a high probability that each 
application will be highly specialized, 
It seems to be more practical to apply 
linear programming techniques to re- 
location problems as they arise. 

Course Costing The major effort required to Implement 

a Course Costing model would be the 
" establishment of a course cost data 
system. Also a technique for course 
costing is currently being developed at 
CNTECHTRA, 

Fleet Training A Fleet Training Requirements Projection 

Requirements Projection model is currently being considered for 

development as part of TRAPS (Training 
Requirements and Planning Subsystem) 
which is one of eight modules of NAVTIS 
(Naval Training Information System). 
Also, because of the problem complexity, 
the risk in constructing a model which 
would accurately represent the Fleet 
training process is rated fairly high. 

Quota Generation Time based projections of required output 

from the training system as a result of 
new equipment or equipment modifications 
are generated primarily by the SYSCOMS 
and OP 29, 39, or 59 and thus falls out- 
side the domain of CNET. The problem of 
quota generation is also related to the 
Fleet Training Requirements Projection 
model in that the data requirements are 
similar. To take advantage of this 
similarity, the models should be developed 
concurrently. 
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Candidate 

Career Path Projection 



Schoolhouse Training vs 
Off-Site Training 

Minimum Level of Training 

Assignment System for 
Self-Paced Study 

PCS Budget Impact on 
Training 

Training Objective Analysis 
NEC Manpower Overage 
Physics of Learning 



Rejected Because 

Although training represents a major 
factor In career path projection, this 
problem falls under the domain of BUPERS 
rather than CNET. The potential benefit 
to be derived by CNET from a Career Path 
Projection model Is questionable. 

These candidates were all eliminated from 
modeling consideration because they 
represent questions that could be answered 
as a result of performing a single evalua- 
tive analysis. Some of these questions 
represent alternatives which could be 
evaluated through application of other 
candidate models, i.e., Educational 
Technology Evaluation model. However, 
the Importance of performing these 
analyses should not be minimized. It is, 
therefore, recommended that these analyses 
be performed within the training organiza- 
tion rather than as part of this contract. 
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CANDIDATE: TRAINING PROCESS FLOW MODEL 



CRITERIA 


RATING 


COMMENT 


PROJECT DEFINITION 


9 


Model represents an R&D effort within 
CNET domain 


ORGANIZATIONAL LEVEL 


7 


Versatility of approach permits application 
at most levels with possible exception of 
CNET 


BENEFIT POTENTIAL 


7 


Estimated highest potential relative to 
remaining six candidate models 


APPLICATION TIME SPAN 


8 


Long-range application anticipated; 
frequency of use rated high 


DEVELOPMENT EFFORT 

APPLICATION/VERIFICATION 
EFFORT 


6 
6 




Medium/low project completion risk; this 
'risk can be controlled through modification 
of scope of model development effort 


TECHNIQUE AVAILABILITY 


7 


Most probable technique will be computer 
simulation; language may be GPSS depending 
on level of detail simulated 


DATA BASE AVAILABILITY 


5 


Portion of data to be generated in parallel 
with model development 


MODEL OBJECTIVE 




Management of training system/training system 
development 


TOTAL RATING 


55 




RANK 


1 


* 



TABLE VII 2 - TRAINING PROCESS FLOW MODEL 
PRELIMINARY EVALUATION 
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CANDIDATE: EDUCATIONAL TECHNOLOGY EVALUATION MODEL 



CRITERIA 


RATING 




PROJECT DEFINITION 


9 


Model represents an R&D effort within CNET 
domain 


ORGANIZATIONAL LEVEL 


7 


Application at functional command and school 
levels; results should impact decisions rela- 

uvc tu cuuv/Q v. i una i tc^finuiuyy Luruuynvju i iicio 


BENEFIT POTENTIAL 


6 


Estimated potential rated 2nd highest relative 
to remaining six candidates 


APPLICATION TIME SPAN 


6 


Model concept has long-range potential; 
algorithms may require updating to reflect 
latest technology 


DEVELOPMENT EFFORT 


6 


Medium risk of completion within project 
time frame 


APPLICATION/ VERIFICATION 
EFFORT 


6 






TECHNIQUE AVAILABILITY 


7 




Application of techniques, algorithms, 
models, and perhaps data from IBM Field 
Engineering experience in field of educa- 
tion lowers risk factor in these areas. 


DATA BASE AVAILA8ILITY 


5 






MODEL OBJECTIVE 




Training system development 


TOTAL RATING 

» 


52 




RANK 


2 





TABLE VII 3 - EDUCATION TECHNOLOGY EVALUATION MODEL 
PRELIMINARY EVALUATION 
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CANDIDATE: SYSTEM CAPABILITY/REQUIREMENTS MODEL 



CRITERIA 


RATING 


COMMENT 


PROJECT DEFINITION 


8 . 


Definitely a CNET application; prior effort 
expended on development of basic approach 
by Navy 


ORGANIZATIONAL LEVEL 


7 


Some use at CNET level; more analysis at 
functional level 


BENEFIT POTENTIAL 


7 


Difficult to assess; facilities and equipment 
oriented - medium/high potential estimated 


APPLICATION TIME SPAN 


"7 
/ 


High probability of long-range application; 
frequency unknown - estimated at low/medium 


ULYLLUrHLIu LFivi\I 


6 


Medium risk of completion within project 
time frame 


APPLICATION/VERIFICATION 
EFFORT 


4 


Difficulty arises from volume of data 

v*pnuHy*pH few* tack 

r Cvj U 1 i CU 1 Ul Lab Pv 


TECHNIQUE AVAILABILITY 


6 


Medium/low risk of applying existing 
techniques 


DATA BASE AVAILABILITY 


5 


Most data available; combination of manual 
and computerized sources; conceptual develop- 
ment of computerized data system exists 


MODEL OBJECTIVE 




Management of training system 


TOTAL RATING 


50 




RANK 


3 





TABLE VII 4 - SYSTEM CAPABILITY/REQUIREMENTS MODEL 
PRELIMINARY EVALUATION 
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CANDIDATE: TRAINING RESOURCE ALLOCATION MODEL 



CRITERIA 


RATING 


COMMENT 


PROJECT DEFINITION 


•8 


Definitely a CNET application. Some efforts 
along these lines already exist 


ORGANIZATIONAL LEVEL 


6 


Application at functional and activity level 


BENEFIT POTENTIAL 


6 


Rated 2nd highest relative to remaining six 
candidates 


APPLICATION TIME SPAN 


7 


High potential of long-range application; 
utilization rate a function of rate of 
training plan modifications 


DEVELOPMENT EFFORT 


6 


Medium risk of completion within project time 
frame 


APPLICATION/VERIFICATION 
EFFORT 


4 


Volume of data to be applied and verified 
increases risk 


TECHNIQUE AVAILABILITY 


6 


Medium/low risk of applying existing technique 


DATA BASE AVAILABILITY 


5 


Most data available: combination of manual 
and computerized sources 


MODEL OBJECTIVE 




Management of training system 


TOTAL RATING 


48 




RANK. 


4 





TABLE VII 5 - TRAINING RESOURCE ALLOCATION MODEL 
PRELIMINARY EVALUATION 
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CANDIDATE: AVIATION TRAINING MANAGEMENT/PLANNING 



CRITERIA 


RATING 


COMMENT 


PROJECT DEFINITION 


c 

0 


Tack fall^ few* pyfpno.'inri anct ^mnpnY/pmpnt" tn 

data base and modeling efforts already 
Initiated at CNATRA 


ORGANIZATIONAL LEVEL 


5 


Application at CNATRA only 

* * 


BENEFIT POTENTIAL 


7 


An effective system has relatively high 
potential because of cost of aviation 
training 


APPLICATION TIME SPAN 


7 


Long-range application projected 


DEVELOPMENT EFFORT 


6 




APPLICATION/VERIFICATION 
EFFORT 


6 


Only a portion of required effort could be 
scheduled for completion during project 
time frame; also a function of telecommunicati 
equipment acquisition 


TECHNIQUE AVAILABILITY 


5 


A function of degree or sophistication of 
models and appl ication 


DATA BASE AVAILABILITY 


6 


Much of data should be available through 
IFTIS 


MODEL OBJECTIVE 




Management of training system 


TOTAL RATING 


48 




RANK 


5 





TABLE VII 6 - AVIATION TRAINING MANAGEMENT/PLANNING MODEL 
PRELIMINARY EVALUATION 
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CANDIDATE: MANAGEMENT OF CONGRESSIONAL AOB CEILINGS 



CRITERIA 


RATING 


COMMENT 


PROJECT OCFINITION 


7 


Mode) should be pursued within CNET; 
however, projected model is not seen to 
Influence training system design 


ORGANIZATIONAL LEVEL 


7 


Primary use at CNET level; potential use 
at lower levels a model design parameter 


ULNFTIT POTENTIAL 


5 


Potential benefit to CNET rated low relative 
to other six candidates 


APPLICATION TIME SPAN 

■ 


6 


Can be utilized for duration of Congressional 
requirement (duration currently unknown) 


DEVELOPMENT EFFORT 


5 


Medium risk of completion within project 
time frame 


APPLICATION/ VERIFICATION 
EFFORT 


5 


Application/verification risk level rated 
equal to development risk 


TECHNIQUE AVAILABILITY 


5 


Medium risk of applying existing techniques; 
portion of data may be available from NITRAS 


DATA BASE AVAILABILITY 


6 


Total data requirements unknown 


MODEL OBJECTIVE 




Management of training system 


TOTAL RATING 


46 




RANK 


6 


; ■" i 



TABLE VI I 7 - MANAGEMENT OF CONGRESSIONAL AOB CEILINGS MODEL 
PRELIMINARY EVALUATION 
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CANDIDATE: BUDGET ANALYSIS MODEL 



CRITERIA 


RATING 


COMMENT 


PROJECT DEFINITION 


4 


Not an R&D effort 


ORGANIZATIONAL LEVEL 


7 


Utilized primarily at DNET 


BENEFIT POTENTIAL 


4 


Lowest relative potential 


APPLICATION TIME SPAN 


5 


Medium-range application 


DEVELOPMENT EFFORT 


1 


Technique development 1s low risk effort 


APPLICATION/ VERIFICATION 
EFFORT 


5 


Application/verification effort rated 
medium risk 


TECHNIQUE AVAILABILITY 


7 


Low risk of applying existing techniques 


DATA BASE AVAILABILITY 


6 


Most data should be available on flat paper 


MODEL OBJECTIVE 




Management of training system 


TOTAL RATING 


45 




RANK 


7 





TABLE VII 8 - BUDGET ANALYSIS MODEL 
PRELIMINARY EVALUATION 
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The preceding evaluations are based on current concepts of potential 
development and application of the candidate models. These evalua- 
tions and the resultant ranking of the candidate models may be 
altered as a result of an analysis of the completed NETS functional 
flow model and as a result of further Investigation and development 
of the top ranked candidates. 

The 11st of seven candidate models was submitted for review by the 
functional commands under CNET. Consents on both candidate model 
rankings and possible areas of application were solicited. 

The response indicated general agreement with the rankings of the 
candidate models. Table VII 9 shows a comparison of the rankings of 
the top seven candidates as 1t appears in the Phase I Report and as 
they were ranked by the functional commands. Since the difference 
in rankings was so slight, those suggested by the functional commands 
can be adopted without further discussion. 



IBM 

1. Training Process Flow 

2. Educational Technology 
Evaluation 

3. System Capabilities/ 
Requirements 

4. Training Resource 
Allocation 

5. Aviation Training 
Management/Planning 

6. Management of 
Congressional AOB 
Ceilings 

7. Budget Analysis 



FUNCTIONAL COMMANDS 

1. Training Process Flow 

2. Educational Technology Evaluation 

3. System Capabilities/Requirements 

4. Training Resource Allocation 

5. Budget Analysis 



6. Aviation Training Management/ 
Planning 



7. Management of Congressional AOB 
Ceilings 



COMPARATIVE MODEL RANKINGS 
TABLE VII 9 
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DESCRIPTION OF SELECTED MODELS 

As a result of the evaluation procedure described 1n Section VII F, 
the following models have been selected for further development: 

o Educational Technology Evaluation 

o Training Process Flow 

o System Capabilities/Requirements 

o Training Resource Allocation. 

The discussion of the model types 1n Section VII C indicates that 
the System Capabilities/Requirements model and the Training Resource 
Allocation model are similar in type, with the Training Resource 
Allocation model addressing a more detailed level of the same basic 
problem. For this reason, the System Capabilities/Requirements model 
and the Training Resource Allocation model will be developed as a 
single model addressing both aspects of the training system output 
vs training system configuration problem. This composite model has 
been designated the System Capabilities/Requirements and Resources 
(SCRR) model. 

All three of the models projected for further development are related 
in that each can provide Inputs to the others. On the other hand, each 
model can be used "stand-alone" to address specific problems at specific 
functional levels. 

Figure VII 3 shows the three model types and their Interrelationships. 
Each of the models operates at a different functional level but eacn 
may be constrained by, or use data produced as output from, one of the 
other models. 

The models are best defined in terms of the functions within the trains 
ing system which they replicate and by whether they deal with Individual 
students, groups of students, or system resources such as Instructors, 
classrooms, or other facilities. The models will be described in terms 
of their internal structure rather than the priority ranking. 

1, Educational Technology Evaluation Model 

The Educational Technology Evaluation model 1s an entity flow 
model. It deals with the movement of Individual students through 
a single course or set of courses. As the name implies, this model 
is concerned with the effect of Instructional strategies and media 
on the flow of students through a course. 

Since it 1s impossible to predict or to represent, mathematically, 
the quantitative or qualitative effect of any learning situation 
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on the student, the model 1s restricted to those attributes of 
the learning situation which can be represented numerically or 
mathematically. 

A model of this type can be constructed to address the following 
learning system attributes: 

a. Media Characteristics 

1) Can the student access the module without assistance^ 
must he do so through some agency as a library? 

2) Is the module self-contained 1n that any remedial 
materials are available within the module? 

3) Are other personnel required either 1n preparation or use 
:>f the module? Other personnel include Instructors, 
support personnel (e.g., computer operators), or other 
students. 

4) Are facilities such as telephone lines, power supplies, 
or specifically equipped carrels required? 

b. Module Selection Process 

Each student 1n the model can be given selected attributes that 
will determine which modules, from the set of available modules 
that student, is required to complete. Further, the model can 
reflect different available media for the same module and 
include media selection rules. 

c. Contention for Facilities 

Students contend for instructors and facilities. Average 
waiting times can be determined as a function of student load, 
equipment type, and number of instructors. 



d. Average Time to Complete 

Average time to complete for different courses or Individual 
modules can be projected for different student loads, media 
mixes, and staffing levels. 

Training Process Flow 

where the Educational Technology Evaluation model deals with the 
movement of individuals through a course or courses, the Training 
Process Flow model deals with the movement of groups of students 
through a collection of courses or schools. The Educational 
Technology Evaluati on model al so 1 s 1 ntended to address courses 
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employing uniform educational strategies whereas the Training 
Process Flow model addresses mixtures of conventional Instruc- 
tion, Individualized Instruction, and mixed lockstep/lndividuallzed. 

Wherever possible, the Training Process Flow model employs known 
or postulated relationships between group attributes and perfor- 
mance. For example, 1f 1t 1s known that students with Basic 
Qualification Test (BQT) scores below a certain level experience 
a fifty percent attrition rate for a particular course, this 
relationship would be employed directly 1n the Training Process 
Flow model 1n projecting course output based on student Input 
characteristics. 

The output of the Training Process Flow model 1s a time-oriented 
profile of training system output 1n terms of numbers of trained 
personnel by type, based upon the characteristics of the Incoming 
students, type of training received, courses available, convening 
dates, etc. This output represents a composite output from a 
number of courses or schools receiving a variety of Input student 
types 1n varying numbers and at varying times. 

System Capabilities/Requirements and Resources Model 

Both the Educational Technology Evaluation model and the Training 
Process Flow model deal directly with students either Individually 
or collectively. The System Capabilities/Requirements and Re- 
sources (SCRR) model 1s oriented around student support require- 
ments in terms of classrooms, laboratories, Instructor billets, 
and course schedules. 

The SCRR model 1s on the same functional level as the Training 
Process Flow model 1n that 1t 1s Intended to deal with a number of 
schools and courses being administered by the same organization, 
but including a variety of course types, class sizes, etc. 

The SCRR model also pisojects system output 1n terms of people by 
type and time. Howeverpits primary purpose 1s to optimize the 
mix of classrooms, instructors, and class schedules needed to produce 
a particular output profile. In the Initial version of the model, 
system output (once optimization takes place) will be calculated 
using class capacity, course length, and convening frequency. 

Note that the SCRR model does not attempt to Include the effects 
of attrition or sttdent Input profile; being concerned ex- 
clusively with available school facilities. 
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Viewed 1n the context of a single, large, complex model, the 
Training Process Flow model performs the same function as the 
system output calculation 1n the SCRR model. However, 1t does 
this projection in a more sophisticated fashion taking Into 
account more variables and more 1nter-relat1onsh1ps. 

Model Interaction 

As previously stated, each of the models can provide either Input 
data or constraint data to the other models. For example: 

a. The Training Process Flow model 1s constrained by the mix of 
courses (length and type) being offered. This Information can 
come from the System Capabilities/Requirements and Resources 
(SCRR) model or the Educational Technology Evaluation model 
depending on the type of planning being done. Essentially, 
the SCRR model determines the configuration of courses and 
classes which the Training Process Flow model will utilize. 

b. The Educational Technology Evaluation model 1s constrained, 1n 
a real -world situation, by the number of Instructors and the 
laboratory space available. These parameters are outputs of 
the SCRR model. 

Either the Training Process Flow model or the SCRR model could 
yield results which would Indicate the desirability of reducing 
a certain common course length to some theoretical level. Such 
a reduced course length could become a constraint 1n the use of 
the Education Technology Evaluation model providing a target 
around which equipment requirements might be optimized. 

c. The System Capabilities/Requirements and Resources (SCRR) 
model requires data as to average time to complete when Indi- 
vidualized instead of conventional lockstep instruction 1s 
used. Projected average time to complete 1s an output of the 
Education Technology Evaluation model. The relationship of 
the SCRR model and the Training Process Flow model has 
already been discussed. 

User Interface 

Successful operational use of any mathematical model depends to a 
great extent on how easy the model is to use. This 1s particularly 
true where operational personnel are rotated and the potential 
user may be unfamiliar with modeling techniques or the models 
themselves. 
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Whether the candidate models are easy to use or not 1s a function 
of two things; the complexity of the model structure and the 
program module designated 1n Figure VII 3 as the user Interface, 

The user Interface program module should provide the following 
kinds of services: 

a. Select which models and which parts of models are to be run. 

b. Specify model constraints such as available equipment, 
number of Instructors, etc. 

c. Update any data file such as equipment Inventory. 

(I. Specify whether optimization 1s to take place (assuming this 
capability exists in the model 1n question). 

While the design of the user Interface module must await detailed 
model specifications, one design goal must be that the Interface 
module provides an Interactive capability to the user. When the 
user specifies changes to the model or Its parameters, he should 
be given verification of the changes requested and, 1f possible, 
Information concerning any other changes to the basic model which 
result from the user request. 
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